-- Logs begin at Wed 2022-05-25 15:30:12 UTC, end at Wed 2022-05-25 17:00:41 UTC. -- May 25 16:08:02.921026 dv-cinder-843279-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 25 16:08:02.930120 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 25 16:08:02 2022] *** May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: nodename: dv-cinder-843279-2-iscsi May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: machine: x86_64 May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: clock source: unix May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: pcre jit disabled May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: detected number of CPU cores: 4 May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: current working directory: / May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: detected binary path: /usr/bin/uwsgi-core May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: your processes number limit is 47680 May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: your memory page size is 4096 bytes May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: detected max file descriptor number: 2048 May 25 16:08:02.932134 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: lock engine: pthread robust mutexes May 25 16:08:02.932942 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: thunder lock: enabled May 25 16:08:02.932942 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 25 16:08:02.932942 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 25 16:08:02.976548 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: Python main interpreter initialized at 0x55f81a9d59c0 May 25 16:08:02.976548 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: python threads support enabled May 25 16:08:02.976548 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: your server socket listen backlog is limited to 100 connections May 25 16:08:02.976548 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: your mercy for graceful operations on workers is 90 seconds May 25 16:08:02.976933 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: mapped 403077 bytes (393 KB) for 2 cores May 25 16:08:02.976933 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: *** Operational MODE: preforking *** May 25 16:08:02.977216 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: *** uWSGI is running in multiple interpreter mode *** May 25 16:08:02.977216 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: spawned uWSGI master process (pid: 103347) May 25 16:08:02.977430 dv-cinder-843279-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 25 16:08:02.977971 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: spawned uWSGI worker 1 (pid: 103348, cores: 1) May 25 16:08:02.978302 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: spawned uWSGI worker 2 (pid: 103349, cores: 1) May 25 16:08:02.978483 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 25 16:08:04.165339 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 25 16:08:04.179931 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 25 16:08:04.897257 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:08:04.897257 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: from cryptography.utils import int_from_bytes May 25 16:08:04.897578 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:08:04.897578 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: from cryptography.utils import int_from_bytes May 25 16:08:04.897700 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:08:04.897700 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: from cryptography.utils import int_from_bytes May 25 16:08:04.897809 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:08:04.897809 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: from cryptography.utils import int_from_bytes May 25 16:08:07.472194 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 16:08:07.472749 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 16:08:07.472841 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 16:08:07.473198 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 16:08:07.473290 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 16:08:07.473378 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 16:08:07.473467 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 16:08:07.473548 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 16:08:07.473833 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.473993 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 16:08:07.474084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 16:08:07.474158 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.474447 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.474579 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.474741 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.474829 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.475079 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.475276 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.475369 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.475440 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.475731 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.475873 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.476012 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.476104 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.476370 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.476516 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.476598 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.476666 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.476930 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.477053 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.477252 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843279-2-iscsi {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.477376 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.477537 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.477642 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.477871 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.478003 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843279-2-iscsi {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.478142 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.478239 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.478480 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.478696 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.478785 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.478858 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.479078 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.479224 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.479383 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.479546 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.479700 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.479866 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.480280 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.480364 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.480501 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.480599 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.480785 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.480901 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.481353 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.481546 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.481725 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.481843 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.482078 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.482249 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.482428 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.482610 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.482717 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.482955 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.483138 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.483321 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.483490 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.483680 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.483846 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.484025 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843279-2-iscsi {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.484193 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.484378 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.484546 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843279-2-iscsi {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.484724 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.484892 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.485108 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.485257 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.485427 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.485597 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.485789 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.485947 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486027 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486092 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486253 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486327 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486475 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486642 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486715 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486780 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486853 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.486930 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.487003 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.487182 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.487320 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.487503 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.487642 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.487769 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.487939 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.488064 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.488229 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.488383 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.488547 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.488674 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.488794 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.488996 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.489122 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.489302 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.489439 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.489558 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.489676 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.489843 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.489967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.490155 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.490289 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.490477 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.490611 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.490785 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.490909 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.491039 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.491221 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.491356 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.491477 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.491603 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.491731 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.491849 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.491969 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.492178 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.492312 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.492446 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.492575 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.492762 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.492887 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.493061 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.493193 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.493350 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.493475 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.493655 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.493780 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.493953 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.494086 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.494212 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.494331 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.494528 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.494654 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.494837 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.494968 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.495125 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.495267 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.495475 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.495602 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.495771 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.495905 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.496085 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.496213 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.45 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.496384 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.45 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.496546 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.45 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.496697 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.45 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.496830 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.497040 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.497171 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.497361 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.497486 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.497669 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.497802 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.497987 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.498112 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.498359 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.498494 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.498622 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.498754 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.498883 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.499001 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.499200 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.499326 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.499513 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.499646 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.499829 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.499964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.500140 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.500270 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.500397 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.500524 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.500643 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.500761 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.500942 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.501067 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.501222 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.501348 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.501546 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.501746 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.501875 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.502002 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.502184 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.502309 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.502469 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.502594 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.502769 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.502893 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.503077 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.503228 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.503390 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.503523 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.503700 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.503909 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.504039 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.504159 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.504277 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.504403 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.504558 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.504692 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.504869 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.504996 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.505167 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.505292 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.505461 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.505597 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.505770 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.505894 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.506075 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.506198 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.506373 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.506505 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.506673 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.506797 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.506977 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.507111 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.507304 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.507434 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.507583 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.507708 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.507885 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.508016 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.508185 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.508318 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.508491 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.508613 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.508784 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.508915 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.509073 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.509207 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.509389 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.509511 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.509687 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.509817 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.509985 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.510119 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.510248 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.510371 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.510568 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.510691 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.510864 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.510996 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.511175 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.511311 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.511487 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.511620 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.511776 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.511908 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.512075 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.512210 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.512360 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.512491 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.512667 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.512790 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.512967 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.513092 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.513267 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.513394 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.513550 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.513685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.513875 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.514000 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.514195 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.514328 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.514504 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.514629 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.514799 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.514929 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.515102 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.515271 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.515391 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.515510 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.515722 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:08:07.515855 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.516048 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.516183 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.516374 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.516500 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.516698 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.516821 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.517003 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.517135 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.517355 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.517490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.517679 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.517804 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.517985 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.518109 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.518286 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.518417 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.518577 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.518700 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.518886 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.519017 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.519211 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.519335 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.519518 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.519642 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.519824 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.519960 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.520138 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.520258 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.520469 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.520601 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.520786 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.520916 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.521101 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.521223 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.521418 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.521539 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.521729 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.521860 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.522032 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.522165 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.522344 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.522467 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.522665 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.522787 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.522975 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.523103 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.523301 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.523435 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.523634 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.523758 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.523968 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.524089 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.524271 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.524473 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.524594 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.524712 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.524891 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.525013 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.525193 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.525322 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.525494 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.525617 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.525797 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.525920 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.526111 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.526249 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.526422 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.526542 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.526725 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.526845 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.527033 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.527200 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.527361 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.527484 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.527663 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.527785 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.527966 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.528091 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.528272 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.528387 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.528607 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.528729 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.528920 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.529043 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.529233 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.529355 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.529542 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.529664 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.529857 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.529980 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.530163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.530286 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.530465 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.530595 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.530766 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.530888 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.531069 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.531238 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.531394 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.531527 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.531694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.531816 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.531989 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.532112 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.532290 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.532420 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.532582 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.532712 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.532910 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.533033 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.533262 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.533392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.533583 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.533704 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.533886 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.534013 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.534196 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.534327 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.534497 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.534625 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.534801 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.534923 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.535104 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.535247 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.535428 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.535548 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.535733 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.535856 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.536032 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.536153 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.536327 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.536448 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.536642 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.536770 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.536944 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.537065 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.537237 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.537358 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.537552 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.537675 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.537849 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.537984 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.538147 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.538269 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.538455 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.538585 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.538755 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.538876 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.539055 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.539222 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.539388 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.539517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.539711 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.539841 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.540024 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.540154 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.540337 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.540475 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.540639 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.540759 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.540966 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.541088 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.541273 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.541397 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.541575 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.541697 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.542259 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.542381 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.542574 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.542703 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.542881 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.543010 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.543207 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.543383 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.543518 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.543645 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.543823 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.543946 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.544128 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.544263 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.544441 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.544571 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.544746 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.544869 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.545052 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.545175 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.545354 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.545475 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.545650 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.545771 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.545944 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.546074 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.546241 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.546364 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.546557 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.546679 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.546859 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.546994 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.547186 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.547323 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.547471 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.547593 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.547767 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.547897 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.548099 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.548229 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.548398 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.548520 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.548757 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.548887 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.549078 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.549198 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.549386 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.549506 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.549700 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.549821 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.550003 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.550124 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.550309 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.550431 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.550613 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.550741 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.550914 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.551036 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.551235 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.551371 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.551540 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.551662 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.551848 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.551968 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.552148 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.552278 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.552455 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.552578 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.552758 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.552881 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.553068 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.553270 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.553391 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.553508 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.553670 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.553801 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.553964 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.554096 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.554270 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.554404 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.554599 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.554758 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.554922 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.555056 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.555249 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.555372 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.555562 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.555696 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.555864 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.555996 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.556166 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.556282 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.556465 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.556586 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.556764 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.556896 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.557063 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.557185 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.557363 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.557485 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.557669 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.557792 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.557968 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.558089 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.558275 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.558395 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.558571 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.558695 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.558882 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.559004 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.559197 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.559322 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.559516 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.559646 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.559826 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.559948 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.560153 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.560274 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.560485 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.560608 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.560796 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.560928 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.561101 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.561230 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.561405 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.561534 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.561714 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.561835 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.562016 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.562153 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.562318 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.562441 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.562619 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.562741 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.562920 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.563070 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.563246 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.563390 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.563551 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.563674 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.563860 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.563990 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.564208 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.564330 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.564517 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.564638 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.564812 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.564932 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.565117 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.565241 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.565418 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.565551 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.565719 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.565844 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.566013 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.566133 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.566314 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.566436 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.566626 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.566748 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.566930 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.567051 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.567263 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.567386 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.567570 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.567692 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.567863 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.567984 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.568159 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.568290 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.568473 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.568601 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.568771 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.568885 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.569065 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.569194 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.569360 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.569490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.569656 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.569779 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.569988 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.570119 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.570290 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.570411 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.570587 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.570714 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.570895 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.571027 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.571226 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.571352 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.571528 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.571663 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.571833 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.571956 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.572133 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.572254 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.572435 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.572557 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.572750 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.572865 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.573058 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.573190 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.573356 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.573477 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.573659 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.573782 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.573962 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.574085 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.574264 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.574387 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.574567 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.574690 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.574878 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.575080 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.575227 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.575442 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.575565 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.575747 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.575878 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.576048 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.576178 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.576346 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.576463 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.576647 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.576953 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.577074 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.577281 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.577511 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.577634 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.577833 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.577969 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.578139 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.578261 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.578450 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.578572 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.578760 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.578883 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.579070 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.579253 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.579420 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.579543 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.579740 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.579938 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.580099 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.580229 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.580399 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.580529 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.580703 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.580832 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.580999 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.581133 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.581296 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.581418 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.581598 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.581719 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.581892 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.582021 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.582190 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.582321 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.582486 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.582609 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.582787 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.582916 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.583086 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.583238 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.583409 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.583532 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.583707 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.583839 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.584003 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.584134 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.584306 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.584429 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.584602 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.584725 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.584905 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.585030 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.585201 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.585323 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.585503 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.585632 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.585803 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.585931 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.586098 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.586228 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.586393 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.586523 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.586686 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.586817 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.587020 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.587142 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.587342 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.587476 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.587670 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.587795 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.587992 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.588114 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.588293 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.588413 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.588610 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.588732 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.588904 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.589026 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.589205 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.589346 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.589504 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.589633 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.589797 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.589919 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.590091 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.590221 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.590387 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.590507 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.590695 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.590816 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.590999 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.591140 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.591329 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.591451 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.591636 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.591758 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.591937 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.592069 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.592242 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.592364 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.592542 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.592663 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.592838 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.592967 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.593139 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.593260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.593436 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.593565 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.593735 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.593857 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.594029 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.594150 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.594324 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.594446 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.594620 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.594742 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.594917 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.595045 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.595229 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.595366 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.595587 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.595709 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.595886 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.596007 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.596186 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.596316 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.596479 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.596600 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.596775 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.596898 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.597070 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.597201 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.597369 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.597490 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.597666 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.597787 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.597962 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.598093 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.598272 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.598441 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.598570 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.598696 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.598850 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.598989 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.599144 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.599283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.599461 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.599592 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.599756 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.599879 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.600049 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.600179 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.600346 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.600468 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.600657 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.600779 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.600962 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.601083 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.601255 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.601378 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.601555 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.601677 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.601852 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.601974 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.602147 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.602270 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.602445 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.602566 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.602739 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.602859 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.603038 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.603229 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.603354 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.603486 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.603640 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.603769 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.603936 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.604066 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.604232 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.604354 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.604535 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.604663 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.604832 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.605032 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.605153 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.605271 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.605409 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.605531 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.605700 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.605835 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.606033 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.606156 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.606371 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.606491 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.606683 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.606818 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.607000 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.607130 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.607337 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.607459 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.607653 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.607777 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.607965 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.608088 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.608268 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.608390 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.608577 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.608699 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.608876 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.608997 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.609188 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.609302 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.609492 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.609615 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.609794 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.609917 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.610090 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.610211 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.610389 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.610512 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.610702 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.610822 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.611023 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.611213 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.611335 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.611464 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.611653 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.611790 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.611970 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.612107 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.612276 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.612404 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.612575 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.612696 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.612880 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.613010 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.613189 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.613315 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.613488 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.613609 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.613793 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.613925 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.614094 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.614214 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.614396 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.614518 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.614701 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.614823 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.615005 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.615127 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.615333 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.615455 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.615635 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.615758 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.615934 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.616055 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.616238 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.616357 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.616535 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.616656 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.616834 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.616955 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.617146 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.617267 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.617449 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.617569 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.617745 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.617865 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.618045 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.618180 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.618343 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.618472 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.618645 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.618774 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.618944 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.619080 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.619414 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.619554 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.619676 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.619794 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.619966 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.620096 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.620263 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.620387 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.620564 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.620686 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.620867 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.621000 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.621166 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.621293 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.621546 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.621667 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.621892 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.622013 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.622204 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.622371 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.622516 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.622681 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.622812 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.622992 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.623115 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.623323 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.623446 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.623626 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.623762 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.623925 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.624047 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.624234 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.624356 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.624554 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.624685 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.624854 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.624977 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.625151 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.625273 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.625451 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.625575 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.625766 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.625888 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.626066 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.626188 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.626379 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.626510 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.626852 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.626972 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.627195 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.627320 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.627478 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.627601 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.627870 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.627990 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.628183 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.628304 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.628492 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.628614 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.628800 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.628921 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.629113 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.629247 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.629419 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.629544 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.629719 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.629849 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.630027 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.630156 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.630330 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.630466 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.630644 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.630773 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.630948 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.631070 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.631272 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.631411 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.631583 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.631706 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.631886 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.632008 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.632202 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.632334 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.632516 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.632638 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.632823 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.632946 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.633144 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.633274 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.633522 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.633642 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.633847 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.633970 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.634149 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.634270 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.634450 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.634570 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.634759 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.634888 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.635060 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.635203 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.635381 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.635504 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.635685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.635807 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.635981 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.636105 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.636280 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.636410 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.636599 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.636721 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.636896 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.637018 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.637198 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.637327 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.637551 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.637681 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.637874 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.637996 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.638178 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.638301 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.638477 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.638606 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.638792 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.638913 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.639095 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.639237 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.639432 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.639554 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.639737 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.639838 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.640028 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.640126 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.640341 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.640438 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.640638 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.640724 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.640941 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.641027 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.641240 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.641328 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.641536 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.641623 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.641832 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.641918 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.642126 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.642222 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.642483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.642581 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.642782 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.642876 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.643098 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.643207 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.643442 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.643555 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.643766 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.643877 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.644066 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.644163 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.644390 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.644479 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.644708 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.644804 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.645011 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.645104 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.645313 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.645401 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.645621 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.645725 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.645913 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.646007 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.646209 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.646296 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.646509 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.646609 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.646804 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.646890 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.647103 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.647266 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.647420 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.647564 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.647721 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.647850 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.648012 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.648132 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.648308 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.648431 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.648602 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.648724 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.648913 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.649036 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.649210 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.649333 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.649509 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.649630 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.649810 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.649931 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.650103 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.650225 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.650401 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.650523 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.650695 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.650828 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.651024 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.651146 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.651342 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.651465 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.651670 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.651809 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.651979 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.652101 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.652294 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.652424 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.652616 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.652750 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.652914 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.653038 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.653226 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.653347 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.653538 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.653670 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.653852 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.653982 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.654165 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.654294 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.654477 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.654607 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.654789 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.654913 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.655103 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.655221 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.655439 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.655529 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.655739 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.655833 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.656045 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.656257 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.656353 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.656586 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.656675 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.656900 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.657143 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.657246 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.657461 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.657562 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.657767 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.657854 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.658119 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.658256 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.658422 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.658519 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.658717 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.658817 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.659020 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.659126 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.659348 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.659436 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.659648 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.659735 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.659955 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.660072 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.660258 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.660354 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.660574 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.660669 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.660887 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.660976 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.661206 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.661302 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.661531 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.661619 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.661839 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.661927 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.662146 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.662235 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.662453 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.662549 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.662759 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.662949 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.663085 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.663210 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.663428 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.663525 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.663730 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.663819 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.664036 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.664133 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.664343 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.664439 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.664648 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.664736 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.664952 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.665039 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.665254 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.665349 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.665560 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.665655 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.665861 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.665947 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.666164 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.666261 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.666465 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.666560 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.667110 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.667234 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.667456 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.667545 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.667774 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.667869 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.668086 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.668181 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.668397 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.668483 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.668707 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.668918 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.669013 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.669235 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.669324 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.669567 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.669806 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.669892 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.670118 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.670227 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.670439 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.670528 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.670760 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.670847 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.671077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.671217 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.671414 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.671502 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.671729 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.671828 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.672039 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.672126 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.672338 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.672424 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.672636 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.672723 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.672945 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.673033 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.673247 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.673346 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.673569 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.673663 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.673883 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.674020 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.674183 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.674263 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.674480 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.674714 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.674801 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.675058 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.675143 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.675406 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.675623 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.675725 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.675962 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.676049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.676274 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.676360 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.676592 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.676688 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.676911 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.677007 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.677225 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.677312 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.677525 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.677620 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.677841 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.677927 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.678141 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.678236 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.678441 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.678536 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.678744 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.678837 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.679046 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.679132 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.679370 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.679459 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.679671 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.679764 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.679972 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.680058 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.680277 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.680376 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.680575 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.680661 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.680877 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.680963 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.681176 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.681263 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.681476 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.681571 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.681784 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.681878 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.682081 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.682169 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.682377 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.682471 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.682681 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.682775 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.682981 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.683067 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.683297 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.683389 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.683597 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.683692 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.683896 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.683990 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.684200 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.684287 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.684512 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.684608 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.684818 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.684907 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.685128 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.685215 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.685477 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.685573 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.685770 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.685856 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.686074 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.686162 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.686382 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.686477 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.686699 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.686786 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.687001 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.687087 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.687339 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.687431 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.687649 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.687736 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.687952 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.688039 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.688269 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.688357 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.688575 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.688662 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.688879 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.689119 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.689297 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.689494 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.689609 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.689817 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.690141 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.690237 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.690471 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.690558 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.690778 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.690865 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.691086 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.691213 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.691420 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.691507 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.691727 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.691813 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.692026 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.692127 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.692330 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.692417 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.692635 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.692722 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.692943 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.693039 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.693249 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.693334 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.693552 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.693637 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.693854 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.693942 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.694154 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.694240 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.694457 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.694552 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.694759 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.694848 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.695061 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.695174 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.695394 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.695493 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.695694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.695781 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.695994 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.696081 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.696294 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.696389 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.696595 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.696681 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.696900 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.696994 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.697235 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.697331 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.697547 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.697634 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.697854 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.697941 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.698158 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.698252 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.698522 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.698619 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.698762 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.698856 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.699064 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.699147 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.699386 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.699473 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.699686 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.699779 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.700007 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.700095 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.700413 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.700501 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.700719 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.700806 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.701019 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.701107 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.701339 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.701435 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.701653 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.701753 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.701980 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.702067 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.702303 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.702390 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.702603 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.702697 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.702921 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.703006 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.703259 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.703371 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.703562 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.703673 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.703873 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.703961 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.704177 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.704264 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.704499 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.704587 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.704859 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.704951 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.705164 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.705252 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.705469 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.705558 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.705770 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.705863 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.706067 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.706154 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.706363 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.706451 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.706662 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.706749 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.706965 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.707064 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.707290 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.707378 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.707591 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.707677 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.707889 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.707988 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.708190 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.708275 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.708486 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.708570 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.708787 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.708884 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.709081 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.709175 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.709377 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.709464 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.709676 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.709770 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.709967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.710053 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.710264 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.710351 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.710559 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.710647 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.710855 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.710942 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.711195 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.711287 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.711470 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.711567 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.711770 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.711858 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.712065 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.712151 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.712363 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.712451 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.712658 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.712743 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.712966 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.713052 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.713261 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.713357 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.713586 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.713673 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.713909 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.714008 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.714214 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.714300 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.714529 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.714615 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.714832 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.714931 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.715146 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.715249 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.715473 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.715559 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.715780 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.715944 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.716111 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.716198 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.716424 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.716512 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.716735 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.716831 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.717047 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.717134 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.717364 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.717460 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.717675 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.717764 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.717978 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.718071 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.718284 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.718378 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.718588 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.718676 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.718891 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.718978 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.719241 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.719332 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.719558 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.719647 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.719855 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.719941 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.720174 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.720271 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.720529 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.720640 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.720870 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.720960 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.721168 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.721267 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.721473 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.721561 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.721787 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.721873 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.722098 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.722236 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.722409 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.722506 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.722726 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.722813 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.723045 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.723133 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.723372 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.723470 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.723682 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.723768 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.723990 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.724085 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.724310 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.724398 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.724628 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.724722 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.724929 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.725016 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.725246 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.725341 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.725567 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.725654 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.725906 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.725993 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.726213 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.726301 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.726525 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.726620 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.726830 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.726917 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.727073 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 16:08:07.727180 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.727480 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.727777 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:08:07.728014 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 16:08:07.785915 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO dbcounter [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Registered counter for database nova_api May 25 16:08:07.786092 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO dbcounter [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Registered counter for database nova_api May 25 16:08:07.808673 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_db.sqlalchemy.engines [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103348) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:08:07.810586 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG oslo_db.sqlalchemy.engines [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103349) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:08:07.815465 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writer thread running {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:08:07.818230 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writer thread running {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:08:07.871416 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.context [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),d6825671-c3c0-4b92-ba50-876e3f2196e8(cell1) {{(pid=103348) load_cells /opt/stack/nova/nova/context.py:464}} May 25 16:08:07.874179 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.context [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),d6825671-c3c0-4b92-ba50-876e3f2196e8(cell1) {{(pid=103349) load_cells /opt/stack/nova/nova/context.py:464}} May 25 16:08:07.875646 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:08:07.877268 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:08:07.878045 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:08:07.878389 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG oslo_concurrency.lockutils [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103349) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:08:07.879992 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG oslo_concurrency.lockutils [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103349) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:08:07.880151 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:08:07.880778 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG oslo_concurrency.lockutils [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103349) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:08:07.881983 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO dbcounter [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Registered counter for database nova_cell0 May 25 16:08:07.882742 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG oslo_concurrency.lockutils [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103349) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:08:07.884761 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO dbcounter [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Registered counter for database nova_cell0 May 25 16:08:07.885984 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO dbcounter [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Registered counter for database nova_cell1 May 25 16:08:07.888654 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO dbcounter [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Registered counter for database nova_cell1 May 25 16:08:07.894984 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_db.sqlalchemy.engines [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103348) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:08:07.899176 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_db.sqlalchemy.engines [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103348) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:08:07.899296 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG oslo_db.sqlalchemy.engines [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103349) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:08:07.900216 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG oslo_db.sqlalchemy.engines [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103349) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:08:07.903416 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writer thread running {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:08:07.903627 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writer thread running {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:08:07.904691 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writer thread running {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:08:08.040653 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writer thread running {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:08:08.047425 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO dbcounter [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Registered counter for database nova_cell0 May 25 16:08:08.051402 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO dbcounter [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Registered counter for database nova_cell0 May 25 16:08:08.056520 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_db.sqlalchemy.engines [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103348) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:08:08.062014 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writer thread running {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:08:08.062571 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG oslo_db.sqlalchemy.engines [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103349) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:08:08.068603 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writer thread running {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:08:08.405155 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: WARNING keystonemiddleware.auth_token [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 25 16:08:08.408889 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: WARNING keystonemiddleware.auth_token [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 25 16:08:08.412463 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: WARNING keystonemiddleware.auth_token [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 25 16:08:08.416166 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: WARNING keystonemiddleware.auth_token [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 25 16:08:08.754815 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: WARNING keystonemiddleware.auth_token [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 25 16:08:08.759616 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: WARNING keystonemiddleware.auth_token [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 25 16:08:08.760823 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: WARNING keystonemiddleware.auth_token [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 25 16:08:08.766109 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f81a9d59c0 pid: 103348 (default app) May 25 16:08:08.766483 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: WARNING keystonemiddleware.auth_token [None req-8206c874-7714-4c4a-a1b9-6153cd3279d1 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 25 16:08:08.768328 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:08:08.769518 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] 10.222.0.45 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001813 May 25 16:08:08.769943 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 788 bytes} [Wed May 25 16:08:08 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:08:08.771660 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f81a9d59c0 pid: 103349 (default app) May 25 16:09:26.725170 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:26.726169 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 May 25 16:09:26.726830 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 2/2] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:26 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:26.738954 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writing DB stats nova_api:SELECT=3 {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:26.740188 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writing DB stats nova_cell0:SELECT=3 {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:26.741191 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writing DB stats nova_cell1:SELECT=3 {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:26.742293 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:27.191211 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-d6a85ad6-95d2-4e7b-8cd3-bde141bb8f22 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:27.221363 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-d6a85ad6-95d2-4e7b-8cd3-bde141bb8f22 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.487655 May 25 16:09:27.222135 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 1/3] 10.222.0.45 () {60 vars in 1291 bytes} [Wed May 25 16:09:26 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:09:29.463866 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:29.465453 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-4febbb98-ca38-42ea-ac2b-8ce5a6870ea6 None None] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002038 May 25 16:09:29.466294 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 3/4] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:29 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:29.526284 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-8211ca68-c1c0-4f41-8a18-a13e3508a205 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "cirros256", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "id": "c1", "swap": 0, "ram": 256}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:29.546414 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:09:29.546414 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: warnings.warn(msg) May 25 16:09:29.575927 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-8211ca68-c1c0-4f41-8a18-a13e3508a205 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.103576 May 25 16:09:29.576587 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 2/5] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:29 2022] POST /compute/v2.1/flavors => generated 407 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:09:29.591411 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_api:SELECT=3 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:29.592589 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_cell0:SELECT=3 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:29.593929 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_cell1:SELECT=3 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:29.594816 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:29.814725 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-78f91c98-6eb8-4844-8d82-36a29ae3aa49 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:29.817379 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:09:29.817379 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: warnings.warn(msg) May 25 16:09:29.882356 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-78f91c98-6eb8-4844-8d82-36a29ae3aa49 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.297416 May 25 16:09:29.889561 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 4/6] 10.222.0.45 () {64 vars in 1335 bytes} [Wed May 25 16:09:29 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:09:32.127090 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-8211ca68-c1c0-4f41-8a18-a13e3508a205 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:32.128199 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-8211ca68-c1c0-4f41-8a18-a13e3508a205 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001549 May 25 16:09:32.128569 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 3/7] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:32 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:32.254982 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-3c03de29-4ad0-44b0-9506-ca2117f210a4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "os-flavor-access:is_public": true, "ram": 512, "rxtx_factor": 1.0, "name": "ds512M", "disk": 5, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "id": "d1"}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:32.273704 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:09:32.273704 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: warnings.warn(msg) May 25 16:09:32.292277 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-3c03de29-4ad0-44b0-9506-ca2117f210a4 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.157213 May 25 16:09:32.293068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 5/8] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:32 2022] POST /compute/v2.1/flavors => generated 404 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:09:32.304993 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-7ec23d23-5a92-4040-bbb1-76a257aa53de admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:32.306448 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:09:32.306448 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: warnings.warn(msg) May 25 16:09:32.337942 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-7ec23d23-5a92-4040-bbb1-76a257aa53de admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.036820 May 25 16:09:32.338518 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 4/9] 10.222.0.45 () {64 vars in 1335 bytes} [Wed May 25 16:09:32 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:09:34.581538 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-3c03de29-4ad0-44b0-9506-ca2117f210a4 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:34.582630 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-3c03de29-4ad0-44b0-9506-ca2117f210a4 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001579 May 25 16:09:34.583395 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 6/10] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:34.649234 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-a482c3ec-7a64-4557-a5b2-85783c2fc711 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "name": "ds1G", "os-flavor-access:is_public": true, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d2", "vcpus": 1, "disk": 10, "rxtx_factor": 1.0}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:34.668922 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-a482c3ec-7a64-4557-a5b2-85783c2fc711 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.079428 May 25 16:09:34.669714 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 5/11] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:34 2022] POST /compute/v2.1/flavors => generated 404 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:09:34.682415 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-1f2e8039-a728-4c24-981f-aa98b550331e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:34.707794 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-1f2e8039-a728-4c24-981f-aa98b550331e admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029145 May 25 16:09:34.708554 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 7/12] 10.222.0.45 () {64 vars in 1335 bytes} [Wed May 25 16:09:34 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:09:36.996158 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-a482c3ec-7a64-4557-a5b2-85783c2fc711 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:36.997186 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-a482c3ec-7a64-4557-a5b2-85783c2fc711 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001843 May 25 16:09:36.997924 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 6/13] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:36 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:37.067559 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-a18cc7da-8c8a-41f6-be73-9d921e943173 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 2, "os-flavor-access:is_public": true, "swap": 0, "name": "ds2G", "rxtx_factor": 1.0, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d3", "ram": 2048}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:37.087707 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-a18cc7da-8c8a-41f6-be73-9d921e943173 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.082926 May 25 16:09:37.088249 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 8/14] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:37 2022] POST /compute/v2.1/flavors => generated 404 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:09:37.100718 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-d19b69e8-50c2-4047-94cf-3702222a097e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:37.126812 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-d19b69e8-50c2-4047-94cf-3702222a097e admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029776 May 25 16:09:37.127481 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 7/15] 10.222.0.45 () {64 vars in 1335 bytes} [Wed May 25 16:09:37 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:09:39.356044 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-a18cc7da-8c8a-41f6-be73-9d921e943173 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:39.357113 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-a18cc7da-8c8a-41f6-be73-9d921e943173 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001605 May 25 16:09:39.358659 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 9/16] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:39 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:39.419745 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-252a6ca8-52cd-4767-8f27-10b7bfb38b79 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 4096, "rxtx_factor": 1.0, "disk": 20, "name": "ds4G", "id": "d4", "swap": 0, "os-flavor-access:is_public": true, "vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:39.438730 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-252a6ca8-52cd-4767-8f27-10b7bfb38b79 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.075249 May 25 16:09:39.442895 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 8/17] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:39 2022] POST /compute/v2.1/flavors => generated 404 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:09:39.451706 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-14205091-1f1c-4f40-80c0-956e012eb194 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:39.476057 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-14205091-1f1c-4f40-80c0-956e012eb194 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028266 May 25 16:09:39.481390 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 10/18] 10.222.0.45 () {64 vars in 1335 bytes} [Wed May 25 16:09:39 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:09:41.703673 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-252a6ca8-52cd-4767-8f27-10b7bfb38b79 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:41.704689 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-252a6ca8-52cd-4767-8f27-10b7bfb38b79 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 May 25 16:09:41.705071 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 9/19] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:41.764012 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-1c8933b7-f4fb-4e2c-9c0a-1430407b3c0a admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:41.787365 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-1c8933b7-f4fb-4e2c-9c0a-1430407b3c0a admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.076173 May 25 16:09:41.788182 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 11/20] 10.222.0.45 () {60 vars in 1291 bytes} [Wed May 25 16:09:41 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:09:44.043682 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-252a6ca8-52cd-4767-8f27-10b7bfb38b79 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:44.044700 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-252a6ca8-52cd-4767-8f27-10b7bfb38b79 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001498 May 25 16:09:44.045220 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 10/21] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:44 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:44.110356 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-5a54d909-5fbb-4bfc-b392-98eaed30fcda admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.tiny", "id": "1", "disk": 1, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "ram": 512}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:44.129578 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-5a54d909-5fbb-4bfc-b392-98eaed30fcda admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.078292 May 25 16:09:44.130146 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 12/22] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:44 2022] POST /compute/v2.1/flavors => generated 402 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:09:44.142381 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-88569ca3-badc-41f2-8f75-6d2d4fa08a3d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:44.168332 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-88569ca3-badc-41f2-8f75-6d2d4fa08a3d admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029675 May 25 16:09:44.168905 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 11/23] 10.222.0.45 () {64 vars in 1332 bytes} [Wed May 25 16:09:44 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:09:46.943610 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-5a54d909-5fbb-4bfc-b392-98eaed30fcda admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:46.944626 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-5a54d909-5fbb-4bfc-b392-98eaed30fcda admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001493 May 25 16:09:46.945031 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 13/24] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:47.006619 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-ebca3192-df2e-4c5a-99e4-ae8448f57032 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "ram": 2048, "id": "2", "os-flavor-access:is_public": true, "disk": 20, "name": "m1.small", "vcpus": 1, "rxtx_factor": 1.0}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:47.026174 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-ebca3192-df2e-4c5a-99e4-ae8448f57032 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.075225 May 25 16:09:47.026734 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 12/25] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:46 2022] POST /compute/v2.1/flavors => generated 405 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:09:47.039883 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-5629d5d7-5636-41e7-a9ab-272e6f94d64d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:47.069020 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-5629d5d7-5636-41e7-a9ab-272e6f94d64d admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033402 May 25 16:09:47.072357 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 14/26] 10.222.0.45 () {64 vars in 1332 bytes} [Wed May 25 16:09:47 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:09:49.282160 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-ebca3192-df2e-4c5a-99e4-ae8448f57032 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:49.283246 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-ebca3192-df2e-4c5a-99e4-ae8448f57032 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001566 May 25 16:09:49.283666 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 13/27] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:49.347725 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-f596700f-b919-4e8d-a3cf-946b6eb25b86 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "3", "disk": 40, "vcpus": 2, "swap": 0, "os-flavor-access:is_public": true, "ram": 4096, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.medium"}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:49.366604 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-f596700f-b919-4e8d-a3cf-946b6eb25b86 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.076516 May 25 16:09:49.367204 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 15/28] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:49 2022] POST /compute/v2.1/flavors => generated 406 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:09:49.379338 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-299f3d5b-f423-41f8-b002-4efb88f14869 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:49.403322 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-299f3d5b-f423-41f8-b002-4efb88f14869 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027885 May 25 16:09:49.404100 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 14/29] 10.222.0.45 () {64 vars in 1332 bytes} [Wed May 25 16:09:49 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:09:51.627110 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-f596700f-b919-4e8d-a3cf-946b6eb25b86 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:51.628895 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-f596700f-b919-4e8d-a3cf-946b6eb25b86 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002292 May 25 16:09:51.629473 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 16/30] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:51 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:51.692066 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-e24acc6f-c9e0-43fd-a596-9ce8669b7a66 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "swap": 0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 8192, "rxtx_factor": 1.0, "disk": 80, "name": "m1.large", "id": "4"}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:51.711236 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-e24acc6f-c9e0-43fd-a596-9ce8669b7a66 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.075180 May 25 16:09:51.711951 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 15/31] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:51 2022] POST /compute/v2.1/flavors => generated 405 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:09:51.724148 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-68ad9a83-4d31-49fa-947b-a6c693f079fd admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:51.747784 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-68ad9a83-4d31-49fa-947b-a6c693f079fd admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027641 May 25 16:09:51.748326 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 17/32] 10.222.0.45 () {64 vars in 1332 bytes} [Wed May 25 16:09:51 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:09:53.989818 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-e24acc6f-c9e0-43fd-a596-9ce8669b7a66 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:09:53.990877 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-e24acc6f-c9e0-43fd-a596-9ce8669b7a66 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001515 May 25 16:09:53.991274 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 16/33] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:09:53 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:09:54.066627 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-dc9e8c74-b697-4d64-b1be-fec6aceaf581 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "ram": 16384, "vcpus": 8, "name": "m1.xlarge", "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 160, "swap": 0, "id": "5", "os-flavor-access:is_public": true}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:54.101846 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-dc9e8c74-b697-4d64-b1be-fec6aceaf581 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.104335 May 25 16:09:54.114895 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 18/34] 10.222.0.45 () {64 vars in 1282 bytes} [Wed May 25 16:09:53 2022] POST /compute/v2.1/flavors => generated 408 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:09:54.121953 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-fac7f049-2a95-4d25-90d9-baceb3ead018 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:09:54.145666 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-fac7f049-2a95-4d25-90d9-baceb3ead018 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029625 May 25 16:09:54.146352 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 17/35] 10.222.0.45 () {64 vars in 1332 bytes} [Wed May 25 16:09:54 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:10:19.295624 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-dc9e8c74-b697-4d64-b1be-fec6aceaf581 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:19.297229 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-dc9e8c74-b697-4d64-b1be-fec6aceaf581 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002309 May 25 16:10:19.297885 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 19/36] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:10:19 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:10:19.308707 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=10,nova_api:SAVEPOINT=5,nova_api:RELEASE=5 {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:19.359255 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-cd3af654-8562-409f-b74d-1f286f045a5b admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:19.376749 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-cd3af654-8562-409f-b74d-1f286f045a5b admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.072757 May 25 16:10:19.377280 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 18/37] 10.222.0.45 () {60 vars in 1291 bytes} [Wed May 25 16:10:19 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:10:20.366900 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-dc9e8c74-b697-4d64-b1be-fec6aceaf581 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:20.367976 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-dc9e8c74-b697-4d64-b1be-fec6aceaf581 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001483 May 25 16:10:20.368403 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 20/38] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:10:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:10:20.581807 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-054f90a4-787b-4e9f-95fa-2b13a944e8ce admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:20.588138 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.wsgi [None req-054f90a4-787b-4e9f-95fa-2b13a944e8ce admin admin] HTTP exception thrown: Flavor 42 could not be found. May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-054f90a4-787b-4e9f-95fa-2b13a944e8ce admin admin] Request method failure captured: May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Accept: application/json May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Accept-Encoding: gzip, deflate May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Connection: keep-alive May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Content-Length: 0 May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Host: 10.222.0.45 May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Openstack-Api-Version: compute 2.90 May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Openstack-System-Scope: None May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: User-Agent: python-novaclient May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Auth-Token: gAAAAABijlTscosXJaJAIPWs_pwWUwwxiCu_fD3-ZXkURdmCEh55ZyPzGRFOKxBxSlOnbIA_qlv8qhKLGGDd4HWKEWQBoMRct4WTMpFikiCo7nN_49WzmJgkd0o7F0u1ucoZSc35RS16XeUHXPGgbDKX03RFM0CWV_JqaVsepQ_rX6IsGWwGfdQ May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Domain-Id: None May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Domain-Name: None May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Identity-Status: Confirmed May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Is-Admin-Project: True May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Openstack-Nova-Api-Version: 2.90 May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Domain-Id: default May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Domain-Name: Default May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Name: admin May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Role: reader,member,admin May 25 16:10:20.591408 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Roles: reader,member,admin May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Tenant: admin May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Tenant-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Tenant-Name: admin May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User: admin May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Domain-Id: default May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Domain-Name: Default May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Id: 32f50b0a3d2643cab337d0af8f4d09eb May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Name: admin May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: method: > May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: exception: Flavor 42 could not be found. May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: action_args: {'id': '42'} May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:10:20.592964 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:20.594125 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:20.595629 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-054f90a4-787b-4e9f-95fa-2b13a944e8ce admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103349) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:10:20.596889 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-054f90a4-787b-4e9f-95fa-2b13a944e8ce admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.085437 May 25 16:10:20.596889 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 19/39] 10.222.0.45 () {60 vars in 1190 bytes} [Wed May 25 16:10:20 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 25 16:10:20.602677 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-27e4addc-046e-42e2-821f-c00e7f28f5f3 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:20.605684 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:20.626065 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-27e4addc-046e-42e2-821f-c00e7f28f5f3 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027005 May 25 16:10:20.626608 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 21/40] 10.222.0.45 () {60 vars in 1210 bytes} [Wed May 25 16:10:20 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:10:20.636449 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-0143e51e-59a8-4b73-88d2-4893e9f0d1ec admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:20.657571 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-0143e51e-59a8-4b73-88d2-4893e9f0d1ec admin admin] 10.222.0.45 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.024822 May 25 16:10:20.658064 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 20/41] 10.222.0.45 () {60 vars in 1210 bytes} [Wed May 25 16:10:20 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:10:20.667758 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-75021bd4-141b-401e-ae9b-9b4fe752847a admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:20.674666 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.wsgi [None req-75021bd4-141b-401e-ae9b-9b4fe752847a admin admin] HTTP exception thrown: Flavor 42 could not be found. May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-75021bd4-141b-401e-ae9b-9b4fe752847a admin admin] Request method failure captured: May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Accept: application/json May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Accept-Encoding: gzip, deflate May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Connection: keep-alive May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Content-Length: 0 May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Host: 10.222.0.45 May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Openstack-Api-Version: compute 2.90 May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Openstack-System-Scope: None May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: User-Agent: python-novaclient May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Auth-Token: gAAAAABijlTscosXJaJAIPWs_pwWUwwxiCu_fD3-ZXkURdmCEh55ZyPzGRFOKxBxSlOnbIA_qlv8qhKLGGDd4HWKEWQBoMRct4WTMpFikiCo7nN_49WzmJgkd0o7F0u1ucoZSc35RS16XeUHXPGgbDKX03RFM0CWV_JqaVsepQ_rX6IsGWwGfdQ May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Domain-Id: None May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Domain-Name: None May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Identity-Status: Confirmed May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Is-Admin-Project: True May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Openstack-Nova-Api-Version: 2.90 May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Domain-Id: default May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Domain-Name: Default May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Name: admin May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Role: reader,member,admin May 25 16:10:20.678077 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Roles: reader,member,admin May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Tenant: admin May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Tenant-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Tenant-Name: admin May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User: admin May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Domain-Id: default May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Domain-Name: Default May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Id: 32f50b0a3d2643cab337d0af8f4d09eb May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Name: admin May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: method: > May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: exception: Flavor 42 could not be found. May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: action_args: {'id': '42'} May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:10:20.679495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:20.680392 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:20.681694 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-75021bd4-141b-401e-ae9b-9b4fe752847a admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103348) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:10:20.682889 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-75021bd4-141b-401e-ae9b-9b4fe752847a admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.015630 May 25 16:10:20.682889 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 22/42] 10.222.0.45 () {60 vars in 1190 bytes} [Wed May 25 16:10:20 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 25 16:10:21.552471 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-0143e51e-59a8-4b73-88d2-4893e9f0d1ec admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:21.553485 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-0143e51e-59a8-4b73-88d2-4893e9f0d1ec admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001438 May 25 16:10:21.553870 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 21/43] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:10:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:10:21.746095 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-76f224b1-e184-4e6a-b36a-bad574d50ccf admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:21.751563 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.wsgi [None req-76f224b1-e184-4e6a-b36a-bad574d50ccf admin admin] HTTP exception thrown: Flavor 84 could not be found. May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-76f224b1-e184-4e6a-b36a-bad574d50ccf admin admin] Request method failure captured: May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Accept: application/json May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Accept-Encoding: gzip, deflate May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Connection: keep-alive May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Content-Length: 0 May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Host: 10.222.0.45 May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Openstack-Api-Version: compute 2.90 May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Openstack-System-Scope: None May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: User-Agent: python-novaclient May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Auth-Token: gAAAAABijlTtt42y2UhP2LVYckgMEK_MNbf8FRG7Aphc0ZU7scG1MGYGKtw57n2LrviBO6g1ksNB01GesxfscJ0LxrgLm8ZLUQ0-Y6bplYqgflpovZN79jOXr09xq_D7OQ3mKAAIHxllZnKvugfwcbGLCtzSYF4nPEe2I7tkDMG2MoaXdHcxTVY May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Domain-Id: None May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Domain-Name: None May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Identity-Status: Confirmed May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Is-Admin-Project: True May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Openstack-Nova-Api-Version: 2.90 May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Domain-Id: default May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Domain-Name: Default May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Name: admin May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Role: reader,member,admin May 25 16:10:21.753084 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Roles: reader,member,admin May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Tenant: admin May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Tenant-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Tenant-Name: admin May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User: admin May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Domain-Id: default May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Domain-Name: Default May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Id: 32f50b0a3d2643cab337d0af8f4d09eb May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Name: admin May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: method: > May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: exception: Flavor 84 could not be found. May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: action_args: {'id': '84'} May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:10:21.754517 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:21.755406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:21.756483 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-76f224b1-e184-4e6a-b36a-bad574d50ccf admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103348) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:10:21.757495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-76f224b1-e184-4e6a-b36a-bad574d50ccf admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.063704 May 25 16:10:21.757495 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 23/44] 10.222.0.45 () {60 vars in 1190 bytes} [Wed May 25 16:10:21 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 25 16:10:21.764578 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-58815a4a-125b-42fa-8066-a43fd816013b admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:21.780006 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-58815a4a-125b-42fa-8066-a43fd816013b admin admin] 10.222.0.45 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019154 May 25 16:10:21.780677 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 22/45] 10.222.0.45 () {60 vars in 1210 bytes} [Wed May 25 16:10:21 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:10:21.789897 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-23e7c242-03a5-4398-85e4-ae92a9f8a756 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:21.804648 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-23e7c242-03a5-4398-85e4-ae92a9f8a756 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018260 May 25 16:10:21.805161 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 24/46] 10.222.0.45 () {60 vars in 1210 bytes} [Wed May 25 16:10:21 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:10:21.816510 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-cff6c902-bbd6-46f3-a210-d607217d2979 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:21.822206 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.wsgi [None req-cff6c902-bbd6-46f3-a210-d607217d2979 admin admin] HTTP exception thrown: Flavor 84 could not be found. May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-cff6c902-bbd6-46f3-a210-d607217d2979 admin admin] Request method failure captured: May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Accept: application/json May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Accept-Encoding: gzip, deflate May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Connection: keep-alive May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Content-Length: 0 May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Host: 10.222.0.45 May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Openstack-Api-Version: compute 2.90 May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Openstack-System-Scope: None May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: User-Agent: python-novaclient May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Auth-Token: gAAAAABijlTtt42y2UhP2LVYckgMEK_MNbf8FRG7Aphc0ZU7scG1MGYGKtw57n2LrviBO6g1ksNB01GesxfscJ0LxrgLm8ZLUQ0-Y6bplYqgflpovZN79jOXr09xq_D7OQ3mKAAIHxllZnKvugfwcbGLCtzSYF4nPEe2I7tkDMG2MoaXdHcxTVY May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Domain-Id: None May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Domain-Name: None May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Identity-Status: Confirmed May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Is-Admin-Project: True May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Openstack-Nova-Api-Version: 2.90 May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Domain-Id: default May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Domain-Name: Default May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Name: admin May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Role: reader,member,admin May 25 16:10:21.823799 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Roles: reader,member,admin May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Tenant: admin May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Tenant-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Tenant-Name: admin May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User: admin May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Domain-Id: default May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Domain-Name: Default May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Id: 32f50b0a3d2643cab337d0af8f4d09eb May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Name: admin May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: method: > May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: exception: Flavor 84 could not be found. May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: action_args: {'id': '84'} May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:10:21.825145 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:21.826049 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:21.827260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-cff6c902-bbd6-46f3-a210-d607217d2979 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103349) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:10:21.828267 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-cff6c902-bbd6-46f3-a210-d607217d2979 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.014352 May 25 16:10:21.828267 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 23/47] 10.222.0.45 () {60 vars in 1190 bytes} [Wed May 25 16:10:21 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 25 16:10:22.722200 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-23e7c242-03a5-4398-85e4-ae92a9f8a756 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:22.723304 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-23e7c242-03a5-4398-85e4-ae92a9f8a756 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001434 May 25 16:10:22.723696 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 25/48] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:10:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:10:22.925808 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-24496888-10fc-43f1-8c05-49a3bb565208 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:22.931974 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.wsgi [None req-24496888-10fc-43f1-8c05-49a3bb565208 admin admin] HTTP exception thrown: Flavor 451 could not be found. May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-24496888-10fc-43f1-8c05-49a3bb565208 admin admin] Request method failure captured: May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Accept: application/json May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Accept-Encoding: gzip, deflate May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Connection: keep-alive May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Content-Length: 0 May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Host: 10.222.0.45 May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Openstack-Api-Version: compute 2.90 May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Openstack-System-Scope: None May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: User-Agent: python-novaclient May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Auth-Token: gAAAAABijlTuu2lseFePlioOPbu7xGV91G6t1uvEywbJZ7nnu3g7R9SHOiewsa7pcwNtHMz5GIU3pbkpqJoqQ3heMtUG6VNixD1jcaWQSgYlgtFBFnKqGtgxA9RnXkf8f-7iFCNkEkum9KZtjkSpCp7FZeulpT7Ca8E2pEmNwJ_C_uMOjU50T9g May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Domain-Id: None May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Domain-Name: None May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Identity-Status: Confirmed May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Is-Admin-Project: True May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Openstack-Nova-Api-Version: 2.90 May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Domain-Id: default May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Domain-Name: Default May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Project-Name: admin May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Role: reader,member,admin May 25 16:10:22.933490 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Roles: reader,member,admin May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Tenant: admin May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Tenant-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-Tenant-Name: admin May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User: admin May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Domain-Id: default May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Domain-Name: Default May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Id: 32f50b0a3d2643cab337d0af8f4d09eb May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: X-User-Name: admin May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: method: > May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: exception: Flavor 451 could not be found. May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: action_args: {'id': '451'} May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:10:22.934966 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:22.936015 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: ERROR nova.api.openstack.wsgi May 25 16:10:22.937283 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-24496888-10fc-43f1-8c05-49a3bb565208 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103349) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:10:22.938686 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-24496888-10fc-43f1-8c05-49a3bb565208 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.074965 May 25 16:10:22.938686 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 24/49] 10.222.0.45 () {60 vars in 1193 bytes} [Wed May 25 16:10:22 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 25 16:10:22.945099 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-1781dd6f-c8b3-4ca1-b7f6-d90f6fa1d0e2 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:22.959234 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-1781dd6f-c8b3-4ca1-b7f6-d90f6fa1d0e2 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017656 May 25 16:10:22.959773 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 26/50] 10.222.0.45 () {60 vars in 1210 bytes} [Wed May 25 16:10:22 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:10:22.969555 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:22.983395 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] 10.222.0.45 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017334 May 25 16:10:22.983915 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 25/51] 10.222.0.45 () {60 vars in 1210 bytes} [Wed May 25 16:10:22 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:10:22.993256 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-36590f7b-d842-42b3-8168-3d61d7f5121f admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:22.999177 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.wsgi [None req-36590f7b-d842-42b3-8168-3d61d7f5121f admin admin] HTTP exception thrown: Flavor 451 could not be found. May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-36590f7b-d842-42b3-8168-3d61d7f5121f admin admin] Request method failure captured: May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Accept: application/json May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Accept-Encoding: gzip, deflate May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Connection: keep-alive May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Content-Length: 0 May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Host: 10.222.0.45 May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Openstack-Api-Version: compute 2.90 May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Openstack-System-Scope: None May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: User-Agent: python-novaclient May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Auth-Token: gAAAAABijlTuu2lseFePlioOPbu7xGV91G6t1uvEywbJZ7nnu3g7R9SHOiewsa7pcwNtHMz5GIU3pbkpqJoqQ3heMtUG6VNixD1jcaWQSgYlgtFBFnKqGtgxA9RnXkf8f-7iFCNkEkum9KZtjkSpCp7FZeulpT7Ca8E2pEmNwJ_C_uMOjU50T9g May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Domain-Id: None May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Domain-Name: None May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Identity-Status: Confirmed May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Is-Admin-Project: True May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Openstack-Nova-Api-Version: 2.90 May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Domain-Id: default May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Domain-Name: Default May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Project-Name: admin May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Role: reader,member,admin May 25 16:10:23.000685 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Roles: reader,member,admin May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e50e9a5bbf7f4ed0b472df8cdb0192de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Tenant: admin May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Tenant-Id: e50e9a5bbf7f4ed0b472df8cdb0192de May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-Tenant-Name: admin May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User: admin May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Domain-Id: default May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Domain-Name: Default May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Id: 32f50b0a3d2643cab337d0af8f4d09eb May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: X-User-Name: admin May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: method: > May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: exception: Flavor 451 could not be found. May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: action_args: {'id': '451'} May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:10:23.002068 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:23.002967 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: ERROR nova.api.openstack.wsgi May 25 16:10:23.004163 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-36590f7b-d842-42b3-8168-3d61d7f5121f admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103348) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:10:23.005193 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-36590f7b-d842-42b3-8168-3d61d7f5121f admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012418 May 25 16:10:23.005193 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 27/52] 10.222.0.45 () {60 vars in 1193 bytes} [Wed May 25 16:10:22 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 25 16:10:23.877322 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:23.878337 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001444 May 25 16:10:23.878744 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 26/53] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:10:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:10:24.078564 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-bde5b03b-b741-4198-8354-ad36c0caf769 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:10:24.101318 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-bde5b03b-b741-4198-8354-ad36c0caf769 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.084865 May 25 16:10:24.101764 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 28/54] 10.222.0.45 () {64 vars in 1235 bytes} [Wed May 25 16:10:24 2022] POST /compute/v2.1/flavors => generated 404 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:10:24.981944 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:24.983261 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001705 May 25 16:10:24.983858 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 27/55] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:10:24 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:10:25.186197 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-c5590695-8513-4aec-8637-586fd4105844 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:10:25.205695 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-c5590695-8513-4aec-8637-586fd4105844 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.079085 May 25 16:10:25.206560 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 29/56] 10.222.0.45 () {64 vars in 1235 bytes} [Wed May 25 16:10:25 2022] POST /compute/v2.1/flavors => generated 405 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:10:26.071264 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:26.072260 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001470 May 25 16:10:26.072642 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 28/57] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:10:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:10:26.281265 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-34c6e1e2-437d-45d7-af69-e24e52e47d1d admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:10:26.302406 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-34c6e1e2-437d-45d7-af69-e24e52e47d1d admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.094285 May 25 16:10:26.302936 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 30/58] 10.222.0.45 () {64 vars in 1235 bytes} [Wed May 25 16:10:26 2022] POST /compute/v2.1/flavors => generated 407 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:10:27.167665 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:27.169153 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002291 May 25 16:10:27.169598 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 29/59] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:10:27 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:10:27.371522 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-dac07e16-2d02-4d9a-8ed3-6fa036125612 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:10:27.393548 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-dac07e16-2d02-4d9a-8ed3-6fa036125612 admin admin] 10.222.0.45 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.086210 May 25 16:10:27.394105 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 31/60] 10.222.0.45 () {64 vars in 1235 bytes} [Wed May 25 16:10:27 2022] POST /compute/v2.1/flavors => generated 409 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:10:28.263932 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:28.264931 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001457 May 25 16:10:28.265321 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 30/61] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:10:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:10:28.455596 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-a91b1683-b736-4fd6-b6cd-9736c440aa3e admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:10:28.474571 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-a91b1683-b736-4fd6-b6cd-9736c440aa3e admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.074048 May 25 16:10:28.475103 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 32/62] 10.222.0.45 () {60 vars in 1202 bytes} [Wed May 25 16:10:28 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:11:30.278152 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:11:30.279275 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-da04f6c9-b8cc-47ca-a95d-e8dcf5298bfe admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001500 May 25 16:11:30.279679 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 31/63] 10.222.0.45 () {54 vars in 930 bytes} [Wed May 25 16:11:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:11:30.289503 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:30.346599 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-5c3cd792-ce51-4fb9-9bd4-1f006e44e478 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:11:30.366297 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-5c3cd792-ce51-4fb9-9bd4-1f006e44e478 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.079399 May 25 16:11:30.366822 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 33/64] 10.222.0.45 () {60 vars in 1291 bytes} [Wed May 25 16:11:30 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:11:30.387925 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG dbcounter [-] [103349] Writing DB stats nova_api:SELECT=14 {{(pid=103349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:30.391895 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-9c983433-7c6e-4f5c-882a-2a812ff42410 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:11:30.400742 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-9c983433-7c6e-4f5c-882a-2a812ff42410 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.014498 May 25 16:11:30.401278 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 32/65] 10.222.0.45 () {60 vars in 1282 bytes} [Wed May 25 16:11:30 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:11:30.413423 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-ca04b802-c572-48ab-9e80-875601e46ce5 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:11:30.421668 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-ca04b802-c572-48ab-9e80-875601e46ce5 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011900 May 25 16:11:30.422185 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 34/66] 10.222.0.45 () {60 vars in 1285 bytes} [Wed May 25 16:11:30 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:11:30.435280 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:11:30.442984 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011199 May 25 16:11:30.443531 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 33/67] 10.222.0.45 () {60 vars in 1285 bytes} [Wed May 25 16:11:30 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:11:30.456181 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-d37c6033-e840-4ff9-b683-d3e7c0c24128 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:11:30.464159 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-d37c6033-e840-4ff9-b683-d3e7c0c24128 admin admin] 10.222.0.45 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011571 May 25 16:11:30.464673 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 35/68] 10.222.0.45 () {60 vars in 1282 bytes} [Wed May 25 16:11:30 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:15:04.158852 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:15:04.159899 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001505 May 25 16:15:04.160291 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 34/69] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:15:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:15:04.314172 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_api:SELECT=6 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:04.443215 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-96c7b031-d6b2-4c40-b8fd-caf34e1e2c33 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:15:04.445168 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:15:04.445168 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: warnings.warn(msg) May 25 16:15:04.448192 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-96c7b031-d6b2-4c40-b8fd-caf34e1e2c33 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:15:04.449206 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-96c7b031-d6b2-4c40-b8fd-caf34e1e2c33 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:15:04.450990 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-96c7b031-d6b2-4c40-b8fd-caf34e1e2c33 admin admin] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:15:04.452022 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-96c7b031-d6b2-4c40-b8fd-caf34e1e2c33 admin admin] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:15:04.487932 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-96c7b031-d6b2-4c40-b8fd-caf34e1e2c33 admin admin] 10.222.0.45 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.175292 May 25 16:15:04.488595 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 36/70] 10.222.0.45 () {60 vars in 1193 bytes} [Wed May 25 16:15:04 2022] GET /compute/v2.1/os-services => generated 791 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:15:05.361138 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:15:05.362158 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001531 May 25 16:15:05.362636 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 35/71] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:15:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:15:05.559827 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-05bbb741-6e91-4a26-ba59-f140fd618345 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:15:05.561364 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:15:05.561364 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: warnings.warn(msg) May 25 16:15:05.563088 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-05bbb741-6e91-4a26-ba59-f140fd618345 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:15:05.563770 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-05bbb741-6e91-4a26-ba59-f140fd618345 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:15:05.564900 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-05bbb741-6e91-4a26-ba59-f140fd618345 admin admin] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:15:05.565487 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-05bbb741-6e91-4a26-ba59-f140fd618345 admin admin] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:15:05.585355 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-05bbb741-6e91-4a26-ba59-f140fd618345 admin admin] 10.222.0.45 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.084603 May 25 16:15:05.585806 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 37/72] 10.222.0.45 () {60 vars in 1193 bytes} [Wed May 25 16:15:05 2022] GET /compute/v2.1/os-services => generated 791 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:15:16.910058 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:15:16.911529 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002464 May 25 16:15:16.912011 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 36/73] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:15:16 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:15:17.058525 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_cell0:SELECT=4 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:17.059421 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_cell1:SELECT=4 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:17.060367 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG dbcounter [-] [103348] Writing DB stats nova_api:SELECT=8 {{(pid=103348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:17.116367 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-ff7bb9cf-b7f9-4aff-b77d-dbf3e238bed3 admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:15:17.117530 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:15:17.117530 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: warnings.warn(msg) May 25 16:15:17.118737 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-ff7bb9cf-b7f9-4aff-b77d-dbf3e238bed3 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:15:17.119193 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-ff7bb9cf-b7f9-4aff-b77d-dbf3e238bed3 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:15:17.119962 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-ff7bb9cf-b7f9-4aff-b77d-dbf3e238bed3 admin admin] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:15:17.120389 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-ff7bb9cf-b7f9-4aff-b77d-dbf3e238bed3 admin admin] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:15:17.144911 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-ff7bb9cf-b7f9-4aff-b77d-dbf3e238bed3 admin admin] 10.222.0.45 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.093845 May 25 16:15:17.145494 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 38/74] 10.222.0.45 () {60 vars in 1193 bytes} [Wed May 25 16:15:17 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:15:18.037376 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: DEBUG nova.api.openstack.wsgi [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] Calling method '>' {{(pid=103349) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:15:18.038396 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: INFO nova.api.openstack.requestlog [None req-bfeecbe3-df87-4450-abcd-65df3e7b1423 admin admin] 10.222.0.45 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001457 May 25 16:15:18.038811 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: [pid: 103349|app: 0|req: 37/75] 10.222.0.45 () {56 vars in 1074 bytes} [Wed May 25 16:15:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:15:18.233403 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG nova.api.openstack.wsgi [None req-21920835-847a-4248-a2c4-7553cdaaf9ac admin admin] Calling method '>' {{(pid=103348) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:15:18.234590 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:15:18.234590 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: warnings.warn(msg) May 25 16:15:18.235849 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-21920835-847a-4248-a2c4-7553cdaaf9ac admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:15:18.236284 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-21920835-847a-4248-a2c4-7553cdaaf9ac admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:15:18.237145 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-21920835-847a-4248-a2c4-7553cdaaf9ac admin admin] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:15:18.237500 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: DEBUG oslo_concurrency.lockutils [None req-21920835-847a-4248-a2c4-7553cdaaf9ac admin admin] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103348) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:15:18.258996 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: INFO nova.api.openstack.requestlog [None req-21920835-847a-4248-a2c4-7553cdaaf9ac admin admin] 10.222.0.45 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.080197 May 25 16:15:18.259561 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: [pid: 103348|app: 0|req: 39/76] 10.222.0.45 () {60 vars in 1193 bytes} [Wed May 25 16:15:18 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:16:16.607263 dv-cinder-843279-2-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... May 25 16:16:16.608745 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: Wed May 25 16:16:16 2022 - graceful shutdown triggered... May 25 16:16:16.609294 dv-cinder-843279-2-iscsi devstack@n-api.service[103348]: Gracefully killing worker 1 (pid: 103348)... May 25 16:16:16.609645 dv-cinder-843279-2-iscsi devstack@n-api.service[103349]: Gracefully killing worker 2 (pid: 103349)... May 25 16:16:17.609999 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: worker 1 buried after 1 seconds May 25 16:16:17.609999 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: worker 2 buried after 1 seconds May 25 16:16:17.609999 dv-cinder-843279-2-iscsi devstack@n-api.service[103347]: goodbye to uWSGI. May 25 16:16:17.612430 dv-cinder-843279-2-iscsi systemd[1]: devstack@n-api.service: Succeeded. May 25 16:16:17.613428 dv-cinder-843279-2-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. May 25 16:16:17.617765 dv-cinder-843279-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 25 16:16:17.641205 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 25 16:16:17 2022] *** May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: nodename: dv-cinder-843279-2-iscsi May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: machine: x86_64 May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: clock source: unix May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: pcre jit disabled May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: detected number of CPU cores: 4 May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: current working directory: / May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: detected binary path: /usr/bin/uwsgi-core May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: your processes number limit is 47680 May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: your memory page size is 4096 bytes May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: detected max file descriptor number: 2048 May 25 16:16:17.668219 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: lock engine: pthread robust mutexes May 25 16:16:17.669244 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: thunder lock: enabled May 25 16:16:17.669244 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 25 16:16:17.669244 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 25 16:16:17.715960 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: Python main interpreter initialized at 0x557eb1d5d9c0 May 25 16:16:17.715960 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: python threads support enabled May 25 16:16:17.715960 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: your server socket listen backlog is limited to 100 connections May 25 16:16:17.715960 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: your mercy for graceful operations on workers is 90 seconds May 25 16:16:17.716439 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: mapped 403077 bytes (393 KB) for 2 cores May 25 16:16:17.716439 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: *** Operational MODE: preforking *** May 25 16:16:17.716654 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: *** uWSGI is running in multiple interpreter mode *** May 25 16:16:17.716654 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: spawned uWSGI master process (pid: 117769) May 25 16:16:17.716876 dv-cinder-843279-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 25 16:16:17.717596 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: spawned uWSGI worker 1 (pid: 117794, cores: 1) May 25 16:16:17.720793 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: spawned uWSGI worker 2 (pid: 117795, cores: 1) May 25 16:16:17.720793 dv-cinder-843279-2-iscsi devstack@n-api.service[117769]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 25 16:16:18.880766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 25 16:16:18.884077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 25 16:16:19.665186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:16:19.665186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: from cryptography.utils import int_from_bytes May 25 16:16:19.665557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:16:19.665557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: from cryptography.utils import int_from_bytes May 25 16:16:19.670719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:16:19.670719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: from cryptography.utils import int_from_bytes May 25 16:16:19.671021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:16:19.671021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: from cryptography.utils import int_from_bytes May 25 16:16:22.076658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 16:16:22.077216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 16:16:22.077429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 16:16:22.077807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 16:16:22.078220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 16:16:22.078419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.078729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.079053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.080041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.080041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.080041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.080375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.080697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.081025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.081349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.081609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.081937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843279-2-iscsi {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.082295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.082602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.082918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.083241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.083546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.083855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.084482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.084664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.086059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.086059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.086059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.086342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.086688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.086962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.087300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.087656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.087939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.088239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.088662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843279-2-iscsi {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.088975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.089288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.089601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.090027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.090342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.090656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.090956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.091307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.091615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.091951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.092269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.092590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.092910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.093211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.093525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.093840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.094161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.094464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.094768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.095018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.095350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.095679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.096001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.096329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.096601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.096884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.097208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.097531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.097849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.098150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.098457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.098763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.099074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.099403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.099545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 16:16:22.100031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.100140 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 16:16:22.100221 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.100292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 16:16:22.100362 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.100609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 16:16:22.100772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.100928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 16:16:22.101088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.101226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.101349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.101550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.101650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.101851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.101945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.45 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.102187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.45 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.102335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.102458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.102546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.102797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.102908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.103113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.103272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.103449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.103551 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.103773 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.103871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.104032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.104151 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.104349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.104463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.104669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843279-2-iscsi {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.104769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.104972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.105081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.105277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.105377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.105583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.105684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.105898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.106002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.106144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.106243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.106445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.106546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.106749 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.106849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.107065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.107305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.107415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.107615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.107714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.107916 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.108019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.108223 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.108315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.108546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.108645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.108850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.108950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.109154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.109243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.109455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.109553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.109761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.109898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.110065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.110157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.110375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.110467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.110680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.110869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843279-2-iscsi {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.110971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.111204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.111301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.111518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.111625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.111832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.111938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.112218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.112315 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.112527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.112621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.112836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.112974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.113157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.113273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.113460 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.113577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.113763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.113871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.114082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.114192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.114386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.114492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.114687 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.114789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.115000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.115101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.115325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.115420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.115629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.115730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.115931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.116037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.116234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.116336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.116532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.116624 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.116833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.116938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.117080 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.117183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.117387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.117546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.117694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.117848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.118009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.118147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.118297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.118436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.118595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.118745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.118892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.119041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.119220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.119359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.119513 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.119678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.119820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.119986 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.120133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.120284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.120440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.120590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.120734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.120881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.121019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.121172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.121313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.121413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.121499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.121582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.121747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.121910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.122013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.122247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.122387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.122537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.122629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.123126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.123347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.123498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.123642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.123791 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.123890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.45 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.123993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.124090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.45 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.124219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.124352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.124472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.124703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.124822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.124941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.125064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.125185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.125311 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.125508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.125631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.125743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.125987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.126169 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.126293 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.126422 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.126642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.126790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.126913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.127033 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.127223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.127372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.127586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.127712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.127833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.128043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.128170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.128289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.128406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.129560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.129708 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.129844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.129951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.129951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.130163 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.130261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.130396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.130493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.130632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.130720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.130837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.130927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.131049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.131143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.131256 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.131380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.131551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.131667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.131782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.131912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.131912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.132117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.132233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.132352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.132469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.132593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.132787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.132938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.133102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.133313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.133404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.133511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.133699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.133865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.133961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.134048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.134279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.134420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.134567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.134730 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.134946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.135057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.135245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.135346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.135563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.135663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.135869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.135964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.136177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.136280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.136476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.136582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.136774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.136865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.137076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.137182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.137379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.137477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.137675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.137796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.137974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.138082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.138274 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.138373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.138575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.138672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.138875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.138978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.139204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.139313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.139495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.139589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.139795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.139894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.140093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.140199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.140390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.140494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.140690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.140789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.140985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.141076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.141298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.141391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.141630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.141720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.141937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.142044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.142264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.142360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.142577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.142677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.142889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.143048 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:16:22.143235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.143347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.143569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.143677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.143874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.143985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.144191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.144282 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.144504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.144596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.144817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.144908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.145127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.145218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.145442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.145543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.145774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.145872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.146076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.146168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.146386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.146477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.146698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.146799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.147010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.147217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.147360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.147497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.147644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.147747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.147974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.148079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.148285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.148390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.148589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.148681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.148901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.149033 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.149205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.149308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.149511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.149611 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.149817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.149917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.150128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.150235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.150434 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.150532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.150740 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.150839 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.151043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.151233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.151369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.151461 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.151672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.151770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.151996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.152089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.152303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.152394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.152606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.152706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.152908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.153138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.153242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.153335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.153539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.153632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.153845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.153945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.154161 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.154264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.154471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.154564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.154780 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.154872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.155089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.155228 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.155425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.155529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.155735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.155835 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.156061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.156192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.156373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.156479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.156683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.156814 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.156991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.157123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.157309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.157442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.158425 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.158605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.158762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.158907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.159082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.159338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.159493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.159637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.159787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.159944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.160089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.160246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.160396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.160548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.160700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.160850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161328 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.161843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.162102 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.162412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.162723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.163030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.163372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.163684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.164002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.164314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.164639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.164948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.165266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.165571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.165887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.166216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.166923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.167260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.167403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.167574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.167750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.167878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.168253 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.168540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.168689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.168828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.169042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.169191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.169389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.169556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.169690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.170004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.170142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.170320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.170507 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.170647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.170919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.171052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.171253 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.171564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.171702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.171868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.172153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.172292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.172467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.172651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.172785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.173091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.173262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.173491 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.173709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.173850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.174015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.174208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.174340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.174632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.174875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.175011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.175314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.175467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.175606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.175914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.176054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.176260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.176506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.176637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.176863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.177003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.177181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.177455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.177586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.177799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.177951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.178120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.178414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.178557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.178779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.178917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.179053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.179366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.179526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.179856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.179996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.180313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.180627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.180757 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.180941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.181155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.181287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.181641 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.181791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.181929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.182169 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.182309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.182480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.182687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.182834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.183084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.183233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.183430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.183720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.184020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.184161 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.184353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.184666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.184815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.184977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.185122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.185299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.185506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.185648 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.185937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.186101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.186248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.186550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.186691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.186907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.187213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.187350 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.187525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.187729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.187873 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.188127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.188271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.188423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.188635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.188783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.189014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.189163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.189327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.189535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.189677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.189922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.190051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.190231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.190509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.190644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.190861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.191047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.191214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.191488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.191762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.191900 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.192105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.192235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.192415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.192612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.192754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.193008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.193153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.193314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.193686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.193829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.193957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.194127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.194259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.194573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.194736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.194959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.195090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.195248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.195529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.195672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.195858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.196164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.196293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.196471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.196665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.196807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.197074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.197205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.197383 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.197592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.197724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.197984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.198115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.198294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.198536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.198746 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.198892 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.199285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.199420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.199548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.199683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.199811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.199960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.200108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.200400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.200535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.200707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.200938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.201085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.201308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.201446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.201619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.201923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.202066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.202235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.202447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.202627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.202840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.202990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.203139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.203406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.203547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.203795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.203988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.204120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.204345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.204485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.204649 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.204902 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.205126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.205265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.205403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.205561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.205836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.205988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.206213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.206492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.206632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.206805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.206987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.207119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.207450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.207594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.207770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.207941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.208084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.208369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.208499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.208673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.208874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.209015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.209293 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.209448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.209593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.209795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.209928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.210154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.210309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.210452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.210745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.210890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.211058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.211267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.211412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.211708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.211973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.212122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.212304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.212491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.212634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.212932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.213066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.213239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.213432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.213601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.213913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.214060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.214230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.214437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.214580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.214845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.214982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.215241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.215437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.215569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.215823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.215957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.216154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.216432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.216574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.216775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.216918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.217082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.217358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.217506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.217701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.217864 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.218010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.218301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.218445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.218631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.218770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.218959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.219244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.219540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.219684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.219827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.219977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.220171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.220303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.220500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.220687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.220829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.221112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.221259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.221420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.221639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.221781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.222079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.222220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.222349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.222662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.222800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.222969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.223199 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.223346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.223600 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.223739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.223905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.224210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.224354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.224534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.224669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.224845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.225156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.225464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.225770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.225953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.226094 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.226407 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.226540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.226810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.226993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.227134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.227464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.227607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.227779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.227979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.228119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.228395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.229101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.229238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.229378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.229687 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.229819 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.229948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.230075 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.230203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.230345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.230474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.230603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.230881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.231022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.231205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.231446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.231639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.231816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.231950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.232216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.232410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.232554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.232736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.232928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.233060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.233349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.233566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.233766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.233898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.234078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.235211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.235361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.235517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.235737 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.235872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.236008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.236008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.236253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.236391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.236520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.236647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.236647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.236896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.237032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.237032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.237335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.237479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.237616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.237902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.238043 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.238218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.238514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.238655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.238823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.239018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.239193 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.239482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.239778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.239925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.240055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.240191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.240426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.240559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.240825 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.240965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.241136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.241366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.241508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.241777 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.241916 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.242084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.242388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.242572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.242705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.242991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.243131 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.243320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.243540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.243681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.243939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.244070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.244250 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.244529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.244662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.244854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.245022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.245236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.245446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.245585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.245823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.245963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.246092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.246353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.246500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.246682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.246965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.247104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.247309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.247494 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.247646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.247918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.248049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.248254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.248402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.248531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.248829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.248965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.249133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.249353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.249487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.249890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.250040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.250171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.250306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.250437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.250707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.250850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.251016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.251215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.251376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.251653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.251842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.252010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.252141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.252416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.252652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.252793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.252988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.253215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.253348 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.253591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.253723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.253901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.254159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.254301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.254497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.254753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.254886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.255098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.255376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.255521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.255719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.255859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.256099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.256239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.256384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.256642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.256781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.256954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.257236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.257368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.257552 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.257735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.257875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.258170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.258310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.258526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.258814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.258955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.259139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.259396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.259537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.259805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.260089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.260230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.260361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.260499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.260743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.260891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.261054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.261227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.261359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.261667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.261810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.261981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.262151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.262283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.262597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.262733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.262893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.263252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.263392 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.263610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.263748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.263931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.264175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.264508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.264649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.264836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.265060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.265194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.265451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.265598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.265764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.265973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.266107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.266379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.266529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.266687 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.266878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.267018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.267317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.267541 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.267682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.267933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.268066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.268255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.268416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.268644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.268912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.269036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.269292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.269608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.269738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.269925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.270223 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.270366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.270543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.270689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.270898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.271303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.271454 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.271632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.271787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.271935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.272133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.272263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.272444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.272757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.273071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.273202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.273386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.273698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.273974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.274106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.274309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.274626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.274859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.275003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.275296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.275621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.275761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.275934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.276249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.276566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.276709 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.277121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.277393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.277534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.277761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.278089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.278227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.278509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.278806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.278939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.279239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.279546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.279679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.279926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.280288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.280433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.280562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.280851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.281052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.281259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.281640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.281779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.281959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.282305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.282447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.282786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.282952 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.283319 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.283530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.283891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.284138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.284301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.284564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.284800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.284933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.285142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.285290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.285493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.285645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.285857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.286079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.286231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.286415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.286662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.286806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.287783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.287950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.288106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.288246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.288385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.288514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.288643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.288771 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.288910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.288910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.289163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.289301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.289301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.289552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.289690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.289833 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.289961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.290089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.290394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.290552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.290685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.290812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.290949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.291234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.291370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.291903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.292143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.292282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.292519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.292654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.292928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.293129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.293260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.293433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.293632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.293766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.294038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.294183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.294352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.294568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.294701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.295015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.295206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.295359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.295544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.295685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.295958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.296099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.296326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.296458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.296587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.296888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.297027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.297198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.297386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.297550 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.297925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.298067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.298299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.298432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.298727 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.298866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.299054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.299256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.299417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.299725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.299918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.300273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.300591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.300979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.301266 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.301413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.301620 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.301798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.301930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.302239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.302377 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.302556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.302771 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.302905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.303219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.303394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.303527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.303786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.303924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.304099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.304320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.304460 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.304706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.304843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.305065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.305242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.305386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.305609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.305750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.305929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.306144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.306283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.306577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.306717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.306877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.307098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.307283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.307530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.307660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.307845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.308001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.308214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.308426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.308559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.308755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.308896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.309075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.309338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.309471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.309714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.309852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.310028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.310361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.310501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.310786 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.310917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.311141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.311307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.311499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.311712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.311845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.312130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.312272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.312458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.312599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.312774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.313148 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.313292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.313431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.313704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.313844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.314051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.314188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.314386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.314717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.314856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.315098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.315270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.315440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.315663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.315808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.315981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.316135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.316340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.316555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.316703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.316895 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.317037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.317261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.317449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.317582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.317793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.317936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.318101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.318352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.318502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.318698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.318842 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.319009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.319275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.319411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.319633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.319763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.319940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.320178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.320312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.320544 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.320698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.320967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.321146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.321284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.321456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.321663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.321806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.322052 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.322182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.322359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.322661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.322803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.322994 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.323284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.323422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.323656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.323810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.323984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.324259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.324392 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.324744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.324921 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.325204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.325359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.325555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.325694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.325875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.326055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.326194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.326500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.326637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.326812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.326995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.327136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.327457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.327600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.327771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.327935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.328087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.328483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.328636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.328771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.328898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.329027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.329325 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.329463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.329654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.329926 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.330068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.330292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.330432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.330612 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.330834 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.331175 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.331312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.331587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.331765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.331987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.332117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.332247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.332491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.332631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.332826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.332972 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.333151 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.334147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.334299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.334299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.334546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.334689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.334826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.334955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.335090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.335297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.335297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.335627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.335758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.335893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.336020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.336020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.336285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.336415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.336624 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.336755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.336953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.337093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.337262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.337421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.337639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 16:16:22.337845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.338217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.338619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.338986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.339390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.339787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.340171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.340543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.340931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.341304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.341693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.342073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.342444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.342857 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.343264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.343683 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.344154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.344545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.344945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.345345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.345728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.346130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.346526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.346936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.347347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.347750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.348114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.348497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.348879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.349248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.349619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.349990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.350365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.350732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.351764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.351764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.352016 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.352253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.352654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.353042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.353401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.353773 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.354232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.354652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.355029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.355461 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.355857 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.356247 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.356643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.357024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.357414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.357868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.358188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.358570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.358953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.359363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.359748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.360130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.360626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.361038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.361570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.361784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.362790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.363085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.363325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.363800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.364141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.364501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.364874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.365248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.365629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.366010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.366387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.366797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.367206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.367545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.367891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.368219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.368535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.368855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.369171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.369535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.369852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.370160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.370476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.370793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.371110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.371447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.371759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.372074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.372388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.372699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.373016 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.373329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.373642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.373970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.374304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.374624 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.374945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.375279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.375594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.376134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.376455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.376794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.377126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.377451 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.377770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.378345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.378674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.378982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.379326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.379640 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.379950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.380253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.380575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.380886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.381240 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.381498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.381811 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.382351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.382704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.383093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.383725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.384327 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.384765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.385078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.385412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.385728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.386296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.386662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.386978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.387296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.387613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.388582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.388582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.388582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.389021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.389232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.389836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.390409 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.390991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.391505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.391865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.392434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.393010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.393653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.394226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.394737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.395060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.395533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.395854 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.396206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.396633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.396864 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.397232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.397570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.397912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.398193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.398560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.398831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.399190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.399503 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.399750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO dbcounter [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Registered counter for database nova_api May 25 16:16:22.399860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.400157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.400479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.401052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.401390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.401718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.402061 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.402397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.402707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.403016 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.403355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.403686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.404003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.404318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.404628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.404939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.405249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.405559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.405870 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.406179 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.406490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.406794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.407102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.407575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.407884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.408200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.408547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.408869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.409213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.409531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.409848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.410161 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.410472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.410783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.411092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.411436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.411772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.412087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.412393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.412822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.413145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.413465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.413786 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.414097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.414681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.414788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.415042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.415389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.415700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.416048 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.416373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.417045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.417372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.417677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.417985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.418288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.418598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.418917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.419267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.419562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.419865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.420059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_db.sqlalchemy.engines [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117795) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:16:22.420211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.420465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.420770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.421073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.421379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.421688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.421992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.422295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.422599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.422990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.423362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.423666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.423975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.424277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.424580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.424937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.425244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.425546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.425896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.426212 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.426525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.426853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.427173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.427508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.427787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writer thread running {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:16:22.427884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.428134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.428459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.428778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.429085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.429423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.429739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.430058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.430376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.430690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.431009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.431352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.431684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.432021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.432326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.432671 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.432986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.433320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.433643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.433953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.434286 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.434597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.434906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.435251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.435560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.435873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.436190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.436504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.436831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.437160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.437470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.437797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.438131 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.438561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.438916 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.439288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.439620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.439934 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.440250 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.440579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:16:22.440820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 16:16:22.484140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.context [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),d6825671-c3c0-4b92-ba50-876e3f2196e8(cell1) {{(pid=117795) load_cells /opt/stack/nova/nova/context.py:464}} May 25 16:16:22.488267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:16:22.489879 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:16:22.490675 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:16:22.492681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:16:22.494664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO dbcounter [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Registered counter for database nova_cell0 May 25 16:16:22.498661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO dbcounter [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Registered counter for database nova_cell1 May 25 16:16:22.499408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO dbcounter [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Registered counter for database nova_api May 25 16:16:22.506116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_db.sqlalchemy.engines [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117795) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:16:22.507642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_db.sqlalchemy.engines [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117795) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:16:22.514636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writer thread running {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:16:22.516176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writer thread running {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:16:22.520625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_db.sqlalchemy.engines [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117794) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:16:22.528082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writer thread running {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:16:22.584766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.context [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),d6825671-c3c0-4b92-ba50-876e3f2196e8(cell1) {{(pid=117794) load_cells /opt/stack/nova/nova/context.py:464}} May 25 16:16:22.588915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:16:22.590527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:16:22.591359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:16:22.593337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:16:22.595366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO dbcounter [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Registered counter for database nova_cell0 May 25 16:16:22.599431 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO dbcounter [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Registered counter for database nova_cell1 May 25 16:16:22.606940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_db.sqlalchemy.engines [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117794) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:16:22.612402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_db.sqlalchemy.engines [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117794) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:16:22.619548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writer thread running {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:16:22.621155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writer thread running {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:16:22.664985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO dbcounter [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Registered counter for database nova_cell0 May 25 16:16:22.672281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_db.sqlalchemy.engines [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117795) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:16:22.677025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writer thread running {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:16:22.770184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO dbcounter [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Registered counter for database nova_cell0 May 25 16:16:22.779234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_db.sqlalchemy.engines [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117794) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:16:22.784782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writer thread running {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:16:22.997856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: WARNING keystonemiddleware.auth_token [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 25 16:16:23.006252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: WARNING keystonemiddleware.auth_token [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 25 16:16:23.105174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: WARNING keystonemiddleware.auth_token [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 25 16:16:23.113038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: WARNING keystonemiddleware.auth_token [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 25 16:16:23.357822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: WARNING keystonemiddleware.auth_token [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 25 16:16:23.363902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: WARNING keystonemiddleware.auth_token [None req-52eae2da-01c6-4db8-9fb9-9504734baf8f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 25 16:16:23.369427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557eb1d5d9c0 pid: 117795 (default app) May 25 16:16:23.454307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: WARNING keystonemiddleware.auth_token [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 25 16:16:23.460423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: WARNING keystonemiddleware.auth_token [None req-00f0e2f1-b9c4-4d08-9a16-e11d7f2d79bc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 25 16:16:23.465688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557eb1d5d9c0 pid: 117794 (default app) May 25 16:17:08.553862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:08.555433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:08.556577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:08.560178 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:09.243846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1327467061", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "networks": [{"uuid": "16379510-80a6-45a8-918e-7b4aff1b4a39"}]}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:10.075376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] validate_networks() for [('16379510-80a6-45a8-918e-7b4aff1b4a39', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:17:12.991244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:17:12.991829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:17:12.992318 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:17:12.993417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:17:12.994033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:17:13.044976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:13.063216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:13.065100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:13.070125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:13.210310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Getting quotas for project fabcb11a7be746719029396d4abab6cc. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:17:13.230264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Getting quotas for user 1cb5512dbadb4b70b030e7e3bb297dad and project fabcb11a7be746719029396d4abab6cc. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:17:13.293472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:17:13.329613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:17:13.561093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2adf9ce8-32cb-4490-a321-39862953dd94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 5.012371 May 25 16:17:13.562085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1/1] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:17:08 2022] POST /compute/v2.1/servers => generated 380 bytes in 5015 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:17:13.578135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=3 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:13.579098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=3 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:13.580095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=3 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:13.581020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=3 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:14.093577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-981d1aba-5e48-43c8-b559-ae481604c632 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:14.094521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-981d1aba-5e48-43c8-b559-ae481604c632 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:15.427379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-981d1aba-5e48-43c8-b559-ae481604c632 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:15.465387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-981d1aba-5e48-43c8-b559-ae481604c632 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1215 microversion: 2.1 time: 1.895736 May 25 16:17:15.467067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1/2] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:13 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1215 bytes in 1899 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:16.486935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7248869c-aeca-47d8-b5ad-2d89fad17bdf tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:16.488131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7248869c-aeca-47d8-b5ad-2d89fad17bdf tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:16.553512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7248869c-aeca-47d8-b5ad-2d89fad17bdf tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:16.556037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7248869c-aeca-47d8-b5ad-2d89fad17bdf tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:16.558684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7248869c-aeca-47d8-b5ad-2d89fad17bdf tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:16.566979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7248869c-aeca-47d8-b5ad-2d89fad17bdf tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:17.283941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7248869c-aeca-47d8-b5ad-2d89fad17bdf tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:17.322801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7248869c-aeca-47d8-b5ad-2d89fad17bdf tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1219 microversion: 2.1 time: 0.841686 May 25 16:17:17.328789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:16 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1219 bytes in 849 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:18.341166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-29b02168-dfbd-45ff-bdf3-206f28814ed7 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:18.341889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-29b02168-dfbd-45ff-bdf3-206f28814ed7 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:18.360565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-29b02168-dfbd-45ff-bdf3-206f28814ed7 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:18.368808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-29b02168-dfbd-45ff-bdf3-206f28814ed7 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:18.369866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-29b02168-dfbd-45ff-bdf3-206f28814ed7 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:18.375043 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-29b02168-dfbd-45ff-bdf3-206f28814ed7 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:19.198810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-29b02168-dfbd-45ff-bdf3-206f28814ed7 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:19.221428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-29b02168-dfbd-45ff-bdf3-206f28814ed7 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1273 microversion: 2.1 time: 0.884192 May 25 16:17:19.227943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:18 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1273 bytes in 892 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:20.245301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-537f395c-a78b-45ba-9866-e61b2ca0a593 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:20.246437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-537f395c-a78b-45ba-9866-e61b2ca0a593 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:20.256375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-537f395c-a78b-45ba-9866-e61b2ca0a593 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:20.256990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-537f395c-a78b-45ba-9866-e61b2ca0a593 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:20.259266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-537f395c-a78b-45ba-9866-e61b2ca0a593 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:20.259995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-537f395c-a78b-45ba-9866-e61b2ca0a593 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:20.561088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-537f395c-a78b-45ba-9866-e61b2ca0a593 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:20.572638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-537f395c-a78b-45ba-9866-e61b2ca0a593 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.331527 May 25 16:17:20.573447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 3/5] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:20 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:21.589830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2c75e01c-ac85-48ca-acf7-590164191e65 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:21.590561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2c75e01c-ac85-48ca-acf7-590164191e65 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:21.603919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c75e01c-ac85-48ca-acf7-590164191e65 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:21.604369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c75e01c-ac85-48ca-acf7-590164191e65 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:21.605163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c75e01c-ac85-48ca-acf7-590164191e65 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:21.605607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c75e01c-ac85-48ca-acf7-590164191e65 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:22.103961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2c75e01c-ac85-48ca-acf7-590164191e65 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:22.119814 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2c75e01c-ac85-48ca-acf7-590164191e65 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.531227 May 25 16:17:22.123969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 3/6] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:21 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:23.002303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-92ff6ebf-6be9-4e9e-9327-df09f8edd204 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Action: 'create', calling method: .version_select at 0x7f4a30d10280>, body: {"keypair": {"name": "tempest-keypair-1511807418", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLPa5/68nZ1aBT7jT7dxHNcQJ6B/AoJShCouqWTmTTgodiamkAmyaNF0fh23nzetIXjt3IIcMcPPaEBLm14/VzXxS9TJp9weRZMHGM8OepUeEJZbKZD2GgnZ3yVF20hIAg=="}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:23.092340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-92ff6ebf-6be9-4e9e-9327-df09f8edd204 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Getting quotas for project b36e59bfe72a49679b3b3657f5a65882. Resources: {'key_pairs'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:17:23.114707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-92ff6ebf-6be9-4e9e-9327-df09f8edd204 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Getting quotas for user e4e4d39c5ed249bc9dc097ba4c24b3d2 and project b36e59bfe72a49679b3b3657f5a65882. Resources: {'key_pairs'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:17:23.153679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:23.307116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-92ff6ebf-6be9-4e9e-9327-df09f8edd204 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.071498 May 25 16:17:23.319540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 4/7] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:17:22 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1080 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:17:23.346290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d2f5bd8a-7f39-4f9b-ad6c-de498f449959 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:23.347106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d2f5bd8a-7f39-4f9b-ad6c-de498f449959 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:23.399076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2f5bd8a-7f39-4f9b-ad6c-de498f449959 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:23.400864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2f5bd8a-7f39-4f9b-ad6c-de498f449959 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:23.402896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2f5bd8a-7f39-4f9b-ad6c-de498f449959 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:23.404149 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2f5bd8a-7f39-4f9b-ad6c-de498f449959 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:23.587776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-963f1908-e465-4777-90af-0187627e7ad9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Action: 'create', calling method: .version_select at 0x7f4a31e09790>, body: {"keypair": {"name": "tempest-keypair-1711374587", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDvpuUO/fffQBvGpPluRAJ6tVjg2rE59crXFfL5ENHbv3k6/60QQWQhU/yDVfJbPhpTHLl/Bite/91y9vEJRwZyxSPDSEhUpf3cmUWWNHHDrI2Tdze9Rt9N8kozBJ1qyAw=="}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:23.809738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-963f1908-e465-4777-90af-0187627e7ad9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Getting quotas for project c3003077416c4ce997e3aa7e47e8e3b7. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:17:23.837480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-963f1908-e465-4777-90af-0187627e7ad9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Getting quotas for user 06ba885a0b164134898c42a332feba76 and project c3003077416c4ce997e3aa7e47e8e3b7. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:17:24.028490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-963f1908-e465-4777-90af-0187627e7ad9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 1.544336 May 25 16:17:24.031217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 4/8] 10.222.0.45 () {62 vars in 1199 bytes} [Wed May 25 16:17:22 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 1550 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) May 25 16:17:24.683371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d2f5bd8a-7f39-4f9b-ad6c-de498f449959 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:24.692300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d2f5bd8a-7f39-4f9b-ad6c-de498f449959 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 1.371387 May 25 16:17:24.708758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 5/9] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:23 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 1388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:25.736973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c8fb8ba6-fc4e-41d8-9a73-f08e582acb94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:25.738560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c8fb8ba6-fc4e-41d8-9a73-f08e582acb94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:25.768763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8fb8ba6-fc4e-41d8-9a73-f08e582acb94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:25.769782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8fb8ba6-fc4e-41d8-9a73-f08e582acb94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:25.777448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8fb8ba6-fc4e-41d8-9a73-f08e582acb94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:25.777890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8fb8ba6-fc4e-41d8-9a73-f08e582acb94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:27.365090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c8fb8ba6-fc4e-41d8-9a73-f08e582acb94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:27.390515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c8fb8ba6-fc4e-41d8-9a73-f08e582acb94 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 1.676838 May 25 16:17:27.399307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 5/10] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:25 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 1692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:28.416946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-799b1da1-645d-4f9f-94ff-584d4b6a0c4a tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:28.417571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-799b1da1-645d-4f9f-94ff-584d4b6a0c4a tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:28.431283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-799b1da1-645d-4f9f-94ff-584d4b6a0c4a tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:28.439449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-799b1da1-645d-4f9f-94ff-584d4b6a0c4a tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:28.441580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-799b1da1-645d-4f9f-94ff-584d4b6a0c4a tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:28.448554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-799b1da1-645d-4f9f-94ff-584d4b6a0c4a tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:29.805501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3481940d-65c1-4416-8f32-72e5253688f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: .version_select at 0x7f4a31d6fd30>, body: {"keypair": {"name": "tempest-keypair-2106604150", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMGAIXE3U8kv3zSIkHJeJFV2abjKOvoTYPu03fOPrRxHBYTH94s7iB6J8TwVRiJ8U8YSZaDBr1HgzczBKp8HN/Uo/or7Cu7mzUoGGvwURGB4FCu+r3kH2JjVjpj8bCQzSA=="}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:29.875761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-3481940d-65c1-4416-8f32-72e5253688f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:17:29.903479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-3481940d-65c1-4416-8f32-72e5253688f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for user a0c9f3f10fd142439521e4b36cf33157 and project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:17:30.054207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3481940d-65c1-4416-8f32-72e5253688f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.744494 May 25 16:17:30.058548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 6/11] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:17:29 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:17:30.473240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-799b1da1-645d-4f9f-94ff-584d4b6a0c4a tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:30.481936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-799b1da1-645d-4f9f-94ff-584d4b6a0c4a tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 2.069260 May 25 16:17:30.486468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 6/12] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:28 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 2074 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:30.758886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-572d8e29-e763-4be4-94a1-3f49c370facb req-d609da55-855d-4e1d-b1e7-0e92e8c9bc8c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "07b98495-1c75-4de3-a9ff-0303003df4e2", "tag": "5883de34-5df8-4619-9f45-34c47caa0f75"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:30.814845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-572d8e29-e763-4be4-94a1-3f49c370facb req-d609da55-855d-4e1d-b1e7-0e92e8c9bc8c service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:30.816927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-572d8e29-e763-4be4-94a1-3f49c370facb req-d609da55-855d-4e1d-b1e7-0e92e8c9bc8c service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:30.902422 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-572d8e29-e763-4be4-94a1-3f49c370facb req-d609da55-855d-4e1d-b1e7-0e92e8c9bc8c service nova] Creating event network-changed:5883de34-5df8-4619-9f45-34c47caa0f75 for instance 07b98495-1c75-4de3-a9ff-0303003df4e2 on hv-cinder-84327 May 25 16:17:30.903974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-572d8e29-e763-4be4-94a1-3f49c370facb req-d609da55-855d-4e1d-b1e7-0e92e8c9bc8c service nova] Acquired lock "compute-rpcapi-router" {{(pid=117794) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 25 16:17:30.913246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5adb2afd-82da-4343-9236-791974ac8b66 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:30.917828 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5adb2afd-82da-4343-9236-791974ac8b66 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:30.925273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5adb2afd-82da-4343-9236-791974ac8b66 None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:30.930290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5adb2afd-82da-4343-9236-791974ac8b66 None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:30.954807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.compute.rpcapi [None req-5adb2afd-82da-4343-9236-791974ac8b66 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 25 16:17:30.959337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5adb2afd-82da-4343-9236-791974ac8b66 None None] Releasing lock "compute-rpcapi-router" {{(pid=117794) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 25 16:17:31.020339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5adb2afd-82da-4343-9236-791974ac8b66 None None] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.799517 May 25 16:17:31.021405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 7/13] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:17:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 802 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:17:31.060614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-145476570", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:31.173245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] validate_networks() for [('729d4051-3bd3-41a9-b052-4f07a89919be', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:17:31.502764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-57a72044-1f73-473c-827b-a1acb5106c12 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:31.504141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-57a72044-1f73-473c-827b-a1acb5106c12 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:31.519032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-57a72044-1f73-473c-827b-a1acb5106c12 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:31.520324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-57a72044-1f73-473c-827b-a1acb5106c12 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:31.522876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-57a72044-1f73-473c-827b-a1acb5106c12 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:31.525891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-57a72044-1f73-473c-827b-a1acb5106c12 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:33.193851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=10 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:33.485717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-57a72044-1f73-473c-827b-a1acb5106c12 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:33.509817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-57a72044-1f73-473c-827b-a1acb5106c12 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 2.006400 May 25 16:17:33.511092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 8/14] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:31 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 2014 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:17:34.528828 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-53289d27-90db-4fbf-9ad7-d4321f8b5a50 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:34.530307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-53289d27-90db-4fbf-9ad7-d4321f8b5a50 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:34.547806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53289d27-90db-4fbf-9ad7-d4321f8b5a50 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:34.548532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53289d27-90db-4fbf-9ad7-d4321f8b5a50 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:34.552155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53289d27-90db-4fbf-9ad7-d4321f8b5a50 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:34.554012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53289d27-90db-4fbf-9ad7-d4321f8b5a50 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:34.651110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:17:34.651583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:17:34.652067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:17:34.652484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:17:34.652985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:17:34.672172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:34.672700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:34.676205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:34.677990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:34.751530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Getting quotas for project bd0c9fdb2e8e4e72ac2835b0482ae824. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:17:34.770792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Getting quotas for user 829c30cd914f48548b987bd6e39ee774 and project bd0c9fdb2e8e4e72ac2835b0482ae824. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:17:34.818200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:17:34.828541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:17:34.977174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-67f3a9f0-026c-42fa-a6ce-847171ce6e9b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.074284 May 25 16:17:34.985774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 7/15] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:17:30 2022] POST /compute/v2.1/servers => generated 383 bytes in 4083 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 25 16:17:34.997179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-285b52d6-d574-4101-b915-cef322664551 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:34.999640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-285b52d6-d574-4101-b915-cef322664551 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:35.715375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-285b52d6-d574-4101-b915-cef322664551 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:35.723841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-53289d27-90db-4fbf-9ad7-d4321f8b5a50 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:35.738071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-53289d27-90db-4fbf-9ad7-d4321f8b5a50 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 1.213135 May 25 16:17:35.739450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 9/16] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:34 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 1216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:35.740062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-285b52d6-d574-4101-b915-cef322664551 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1221 microversion: 2.1 time: 0.752939 May 25 16:17:35.745136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 8/17] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:34 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1221 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:36.135901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1405565171", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:36.304115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.network.neutron [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] validate_networks() for [('729d4051-3bd3-41a9-b052-4f07a89919be', None, None, None, None, None)] {{(pid=117794) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:17:36.392414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1615973321", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "28311134-63e6-4448-9485-57f8501da8ed"}], "security_groups": [{"name": "tempest-securitygroup--179594977"}], "key_name": "tempest-keypair-1511807418", "user_data": "}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:37.282363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] validate_networks() for [('28311134-63e6-4448-9485-57f8501da8ed', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:17:38.520881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Flavor limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:17:38.522673 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Image limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:17:38.524835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Flavor pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:17:38.525781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Image pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:17:38.526500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:17:38.546420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:38.547386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:38.549206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:38.550515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:38.583656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Getting quotas for project bd0c9fdb2e8e4e72ac2835b0482ae824. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:17:38.592164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Getting quotas for user 829c30cd914f48548b987bd6e39ee774 and project bd0c9fdb2e8e4e72ac2835b0482ae824. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:17:38.610958 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Going to run 1 instances... {{(pid=117794) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:17:38.615203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117794) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:17:38.735438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-85b24c6a-89f1-473b-b760-7abfbf6cab2e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.604497 May 25 16:17:38.739824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 10/18] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:17:36 2022] POST /compute/v2.1/servers => generated 383 bytes in 2611 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 25 16:17:38.749135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4e424b4f-a7a4-4dc8-b8f5-68c0373cd14b tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:38.750712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4e424b4f-a7a4-4dc8-b8f5-68c0373cd14b tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:38.764012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4e424b4f-a7a4-4dc8-b8f5-68c0373cd14b tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:38.764790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4e424b4f-a7a4-4dc8-b8f5-68c0373cd14b tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:38.765942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4e424b4f-a7a4-4dc8-b8f5-68c0373cd14b tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:38.767209 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4e424b4f-a7a4-4dc8-b8f5-68c0373cd14b tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:38.834289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:17:38.835338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:17:38.835974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:17:38.836562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:17:38.837130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:17:38.839256 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=54,nova_api:INSERT=7 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:38.856528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:38.856997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:38.857810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:38.858260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:38.885790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=29 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:38.911037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Getting quotas for project b36e59bfe72a49679b3b3657f5a65882. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:17:38.915240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Getting quotas for user e4e4d39c5ed249bc9dc097ba4c24b3d2 and project b36e59bfe72a49679b3b3657f5a65882. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:17:38.949113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:17:38.951472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:17:39.037942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a93e92f6-76db-4f50-8115-40ed156e009e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 2.650064 May 25 16:17:39.042627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 9/19] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:17:36 2022] POST /compute/v2.1/servers => generated 401 bytes in 2655 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:17:39.050072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-116848418", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d4194a59-8440-4e29-8e43-bfe2dcadce67"}], "security_groups": [{"name": "tempest-securitygroup--1229336644"}], "key_name": "tempest-keypair-1711374587", "user_data": "}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:39.081368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4e424b4f-a7a4-4dc8-b8f5-68c0373cd14b tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:39.100856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4e424b4f-a7a4-4dc8-b8f5-68c0373cd14b tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.354689 May 25 16:17:39.102374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 11/20] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:38 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:39.108366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-641021200", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "aaf08a79-3442-476f-b613-a02c12c36d8d"}], "security_groups": [{"name": "tempest-securitygroup--2121589677"}], "key_name": "tempest-keypair-2106604150", "user_data": "}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:39.478662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.network.neutron [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] validate_networks() for [('aaf08a79-3442-476f-b613-a02c12c36d8d', None, None, None, None, None)] {{(pid=117794) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:17:39.512286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] validate_networks() for [('d4194a59-8440-4e29-8e43-bfe2dcadce67', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:17:41.704803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:17:41.706010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:17:41.706665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:17:41.707312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:17:41.708290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:17:41.738718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:41.741729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:41.743930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:41.746606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:41.794860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Getting quotas for project c3003077416c4ce997e3aa7e47e8e3b7. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:17:41.800088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Getting quotas for user 06ba885a0b164134898c42a332feba76 and project c3003077416c4ce997e3aa7e47e8e3b7. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:17:41.824141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:17:41.826887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:17:41.906826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2c524395-e673-41f4-9526-dcdb780b2b90 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 2.861910 May 25 16:17:41.908155 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 10/21] 10.222.0.45 () {62 vars in 1187 bytes} [Wed May 25 16:17:39 2022] POST /compute/v2.1/servers => generated 402 bytes in 2865 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) May 25 16:17:41.915461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-677e7f79-3db5-496b-8198-64aba2bb1c08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:41.917911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-677e7f79-3db5-496b-8198-64aba2bb1c08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:41.928174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-677e7f79-3db5-496b-8198-64aba2bb1c08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:41.928863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-677e7f79-3db5-496b-8198-64aba2bb1c08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:41.931476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-677e7f79-3db5-496b-8198-64aba2bb1c08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:41.936123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-677e7f79-3db5-496b-8198-64aba2bb1c08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:42.153247 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Flavor limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:17:42.154679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Image limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:17:42.155193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Flavor pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:17:42.155633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Image pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:17:42.156234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:17:42.180341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:42.180803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:42.182538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:42.182987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:42.241570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:17:42.252625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for user a0c9f3f10fd142439521e4b36cf33157 and project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:17:42.279128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Going to run 1 instances... {{(pid=117794) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:17:42.281992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117794) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:17:42.304281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-677e7f79-3db5-496b-8198-64aba2bb1c08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:42.317689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-677e7f79-3db5-496b-8198-64aba2bb1c08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1280 microversion: 2.1 time: 0.406242 May 25 16:17:42.320416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 11/22] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:41 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1280 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:42.332701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b28e857f-0cb8-406a-98ac-8ebdd1470012 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:42.333306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b28e857f-0cb8-406a-98ac-8ebdd1470012 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:42.347512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b28e857f-0cb8-406a-98ac-8ebdd1470012 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:42.347974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b28e857f-0cb8-406a-98ac-8ebdd1470012 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:42.348757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b28e857f-0cb8-406a-98ac-8ebdd1470012 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:42.349213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b28e857f-0cb8-406a-98ac-8ebdd1470012 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:42.420060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2fc7ac4f-a879-482b-baaa-ae95ed370672 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.316378 May 25 16:17:42.425450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 12/23] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:17:39 2022] POST /compute/v2.1/servers => generated 402 bytes in 3322 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:17:42.439005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5662988f-8ceb-4c11-8cb1-85142f478a05 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:42.450484 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5662988f-8ceb-4c11-8cb1-85142f478a05 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:42.463522 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5662988f-8ceb-4c11-8cb1-85142f478a05 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:42.464618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5662988f-8ceb-4c11-8cb1-85142f478a05 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:42.465597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5662988f-8ceb-4c11-8cb1-85142f478a05 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:42.466440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5662988f-8ceb-4c11-8cb1-85142f478a05 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:42.910395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b28e857f-0cb8-406a-98ac-8ebdd1470012 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:42.927325 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b28e857f-0cb8-406a-98ac-8ebdd1470012 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1296 microversion: 2.1 time: 0.605006 May 25 16:17:42.930536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 12/24] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:42 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1296 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:42.942277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:42.945248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:42.953715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:42.954384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:42.956035 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:42.956719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:43.027171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5662988f-8ceb-4c11-8cb1-85142f478a05 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:43.040446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5662988f-8ceb-4c11-8cb1-85142f478a05 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.610772 May 25 16:17:43.052362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 13/25] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:42 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:43.061107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:43.066699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-21bb5180-454c-4630-a425-0b9a3249cc2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:43.067401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-21bb5180-454c-4630-a425-0b9a3249cc2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:43.075752 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:43.077316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:43.078546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-30eec992-5bf4-48a8-89e3-4dd640800f5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1277 microversion: 2.20 time: 0.145692 May 25 16:17:43.079114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 13/26] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1277 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:43.258754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-9d115228-5629-40d4-947c-9bca4820d5a5 req-dff3644d-0c4f-41a3-8d03-b1bacad0438a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "69fc8c5b-82f0-4a03-8dab-381fdee5884d", "tag": "4220f323-95a7-4c14-8c3e-42c4591d25b3"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:43.272294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21bb5180-454c-4630-a425-0b9a3249cc2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:43.280273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-21bb5180-454c-4630-a425-0b9a3249cc2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1241 microversion: 2.1 time: 0.226220 May 25 16:17:43.292460 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-9d115228-5629-40d4-947c-9bca4820d5a5 req-dff3644d-0c4f-41a3-8d03-b1bacad0438a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:43.293211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-9d115228-5629-40d4-947c-9bca4820d5a5 req-dff3644d-0c4f-41a3-8d03-b1bacad0438a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:43.294237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 14/27] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:43 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1241 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:43.320798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-9d115228-5629-40d4-947c-9bca4820d5a5 req-dff3644d-0c4f-41a3-8d03-b1bacad0438a service nova] Creating event network-changed:4220f323-95a7-4c14-8c3e-42c4591d25b3 for instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d on hv-cinder-84327 May 25 16:17:43.321344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-9d115228-5629-40d4-947c-9bca4820d5a5 req-dff3644d-0c4f-41a3-8d03-b1bacad0438a service nova] Acquired lock "compute-rpcapi-router" {{(pid=117795) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 25 16:17:43.324277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ad6a8ed-02ec-4ab1-9f0b-0d00499299f6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:43.324737 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ad6a8ed-02ec-4ab1-9f0b-0d00499299f6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:43.325556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ad6a8ed-02ec-4ab1-9f0b-0d00499299f6 None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:43.326156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ad6a8ed-02ec-4ab1-9f0b-0d00499299f6 None None] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:43.339480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1fe714c1-5bcf-4944-a118-edf84f2ce72a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:43.340236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1fe714c1-5bcf-4944-a118-edf84f2ce72a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:43.352143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fe714c1-5bcf-4944-a118-edf84f2ce72a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:43.352990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fe714c1-5bcf-4944-a118-edf84f2ce72a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:43.355203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.compute.rpcapi [None req-9ad6a8ed-02ec-4ab1-9f0b-0d00499299f6 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 25 16:17:43.355695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fe714c1-5bcf-4944-a118-edf84f2ce72a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:43.356240 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fe714c1-5bcf-4944-a118-edf84f2ce72a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:43.356700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ad6a8ed-02ec-4ab1-9f0b-0d00499299f6 None None] Releasing lock "compute-rpcapi-router" {{(pid=117795) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 25 16:17:43.393833 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9ad6a8ed-02ec-4ab1-9f0b-0d00499299f6 None None] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.139526 May 25 16:17:43.396525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 14/28] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:17:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:17:43.827540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1fe714c1-5bcf-4944-a118-edf84f2ce72a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:43.836372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1fe714c1-5bcf-4944-a118-edf84f2ce72a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.501908 May 25 16:17:43.847134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 15/29] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:43 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:17:43.948471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c23f1bce-e28b-4fe4-a1de-88bdd0cad12b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:43.949285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c23f1bce-e28b-4fe4-a1de-88bdd0cad12b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:44.004468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c23f1bce-e28b-4fe4-a1de-88bdd0cad12b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:44.004992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c23f1bce-e28b-4fe4-a1de-88bdd0cad12b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:44.009818 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c23f1bce-e28b-4fe4-a1de-88bdd0cad12b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:44.011467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c23f1bce-e28b-4fe4-a1de-88bdd0cad12b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:44.060590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-802a7431-35e6-4f96-a827-cc6ad949ca1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:44.061379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-802a7431-35e6-4f96-a827-cc6ad949ca1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:44.069394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-802a7431-35e6-4f96-a827-cc6ad949ca1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:44.088521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-802a7431-35e6-4f96-a827-cc6ad949ca1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:44.088521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-802a7431-35e6-4f96-a827-cc6ad949ca1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:44.088521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-802a7431-35e6-4f96-a827-cc6ad949ca1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:45.191390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c23f1bce-e28b-4fe4-a1de-88bdd0cad12b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:45.199599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c23f1bce-e28b-4fe4-a1de-88bdd0cad12b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 1.256004 May 25 16:17:45.203596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 15/30] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:43 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 1261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:45.217138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-802a7431-35e6-4f96-a827-cc6ad949ca1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:45.219599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=61 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:45.221390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:45.222391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:45.227030 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-802a7431-35e6-4f96-a827-cc6ad949ca1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 1.171205 May 25 16:17:45.233446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 16/31] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:44 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 1178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:45.242342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:45.242808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:45.243622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:45.248084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:45.334382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:45.354014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:45.357146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:45.358472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2056e71d-fb75-45bf-b67b-8b36a334c1b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1339 microversion: 2.20 time: 0.153672 May 25 16:17:45.359396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 16/32] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:45 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1339 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:45.364809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f06b0431-5f10-439f-bfaf-174e9a1ac1f2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:45.365727 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f06b0431-5f10-439f-bfaf-174e9a1ac1f2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:45.372469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f06b0431-5f10-439f-bfaf-174e9a1ac1f2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:45.373116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f06b0431-5f10-439f-bfaf-174e9a1ac1f2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:45.374231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f06b0431-5f10-439f-bfaf-174e9a1ac1f2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:45.374807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f06b0431-5f10-439f-bfaf-174e9a1ac1f2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:45.479616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f06b0431-5f10-439f-bfaf-174e9a1ac1f2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:45.491392 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f06b0431-5f10-439f-bfaf-174e9a1ac1f2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1299 microversion: 2.1 time: 0.130750 May 25 16:17:45.493223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 17/33] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:45 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1299 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:45.498438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0b6061af-48f6-4967-a3cd-51adc5a4c46b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:45.500093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0b6061af-48f6-4967-a3cd-51adc5a4c46b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:45.507198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0b6061af-48f6-4967-a3cd-51adc5a4c46b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:45.507924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0b6061af-48f6-4967-a3cd-51adc5a4c46b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:45.510266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0b6061af-48f6-4967-a3cd-51adc5a4c46b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:45.510725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0b6061af-48f6-4967-a3cd-51adc5a4c46b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:45.572768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-58ad49f9-359a-4a84-8f2c-b495ce486d54 req-f6d5b0f6-3a0b-4f81-aeb7-898a1a685747 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "07b98495-1c75-4de3-a9ff-0303003df4e2", "name": "network-vif-plugged", "status": "completed", "tag": "5883de34-5df8-4619-9f45-34c47caa0f75"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:45.582514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=79,nova_api:INSERT=8 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:45.599099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-58ad49f9-359a-4a84-8f2c-b495ce486d54 req-f6d5b0f6-3a0b-4f81-aeb7-898a1a685747 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:45.599756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-58ad49f9-359a-4a84-8f2c-b495ce486d54 req-f6d5b0f6-3a0b-4f81-aeb7-898a1a685747 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:45.629300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-58ad49f9-359a-4a84-8f2c-b495ce486d54 req-f6d5b0f6-3a0b-4f81-aeb7-898a1a685747 service nova] Creating event network-vif-plugged:5883de34-5df8-4619-9f45-34c47caa0f75 for instance 07b98495-1c75-4de3-a9ff-0303003df4e2 on hv-cinder-84327 May 25 16:17:45.642281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-58ad49f9-359a-4a84-8f2c-b495ce486d54 req-f6d5b0f6-3a0b-4f81-aeb7-898a1a685747 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.407361 May 25 16:17:45.646561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 17/34] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:17:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:17:45.738277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0b6061af-48f6-4967-a3cd-51adc5a4c46b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:45.749528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0b6061af-48f6-4967-a3cd-51adc5a4c46b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.254890 May 25 16:17:45.750113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 18/35] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:45 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:45.861244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-86e5e0c6-47f6-4cc6-9284-a236707436f6 req-befb955e-b08f-4012-9497-f841430ea852 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f858e1ab-1766-4273-be90-d3c73f8678da", "tag": "dc3d60cd-b59c-445a-84d9-b0658a3a8344"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:45.876239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-86e5e0c6-47f6-4cc6-9284-a236707436f6 req-befb955e-b08f-4012-9497-f841430ea852 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:45.877213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-86e5e0c6-47f6-4cc6-9284-a236707436f6 req-befb955e-b08f-4012-9497-f841430ea852 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:45.893819 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-86e5e0c6-47f6-4cc6-9284-a236707436f6 req-befb955e-b08f-4012-9497-f841430ea852 service nova] Creating event network-changed:dc3d60cd-b59c-445a-84d9-b0658a3a8344 for instance f858e1ab-1766-4273-be90-d3c73f8678da on hv-cinder-84327 May 25 16:17:45.900341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-86e5e0c6-47f6-4cc6-9284-a236707436f6 req-befb955e-b08f-4012-9497-f841430ea852 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.181014 May 25 16:17:45.901564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 18/36] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:17:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:17:46.219532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b03b09fe-d103-4e02-84ae-877e458c43f3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:46.220142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b03b09fe-d103-4e02-84ae-877e458c43f3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:46.226292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b03b09fe-d103-4e02-84ae-877e458c43f3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.226738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b03b09fe-d103-4e02-84ae-877e458c43f3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.227916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b03b09fe-d103-4e02-84ae-877e458c43f3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.228475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b03b09fe-d103-4e02-84ae-877e458c43f3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.244568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-caf650f8-733c-4adc-a8e9-16e76b9eabe2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:46.246169 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-caf650f8-733c-4adc-a8e9-16e76b9eabe2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:46.254815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-caf650f8-733c-4adc-a8e9-16e76b9eabe2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.255313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-caf650f8-733c-4adc-a8e9-16e76b9eabe2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.256124 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-caf650f8-733c-4adc-a8e9-16e76b9eabe2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.256566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-caf650f8-733c-4adc-a8e9-16e76b9eabe2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.432347 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b03b09fe-d103-4e02-84ae-877e458c43f3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:46.442976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b03b09fe-d103-4e02-84ae-877e458c43f3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.226490 May 25 16:17:46.444206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 19/37] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:46 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:46.449715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:46.450590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:46.458150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.458852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.459865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.460918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.479715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-caf650f8-733c-4adc-a8e9-16e76b9eabe2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:46.487436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-caf650f8-733c-4adc-a8e9-16e76b9eabe2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.247045 May 25 16:17:46.488550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 19/38] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:46 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:17:46.511014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d4d4b10d-e185-463a-afa6-96abcf1ea9bd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:46.511601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d4d4b10d-e185-463a-afa6-96abcf1ea9bd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:46.517646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4d4b10d-e185-463a-afa6-96abcf1ea9bd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.518284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4d4b10d-e185-463a-afa6-96abcf1ea9bd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.519052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4d4b10d-e185-463a-afa6-96abcf1ea9bd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.519688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4d4b10d-e185-463a-afa6-96abcf1ea9bd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.574285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:46.583450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:46.584956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:46.586064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-314a72bd-3510-4c0c-ad60-a66e692b4e20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1339 microversion: 2.20 time: 0.140611 May 25 16:17:46.586932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 20/39] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:46 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1339 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:46.609174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d4d4b10d-e185-463a-afa6-96abcf1ea9bd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:46.617724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d4d4b10d-e185-463a-afa6-96abcf1ea9bd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1299 microversion: 2.1 time: 0.110450 May 25 16:17:46.618546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 20/40] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:46 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1299 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:17:46.765733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e0afed66-ebb0-4b66-bc27-29eabbf6f3b5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:46.766349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e0afed66-ebb0-4b66-bc27-29eabbf6f3b5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:46.772706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0afed66-ebb0-4b66-bc27-29eabbf6f3b5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.773156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0afed66-ebb0-4b66-bc27-29eabbf6f3b5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.776956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0afed66-ebb0-4b66-bc27-29eabbf6f3b5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:46.777438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0afed66-ebb0-4b66-bc27-29eabbf6f3b5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:46.952088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e0afed66-ebb0-4b66-bc27-29eabbf6f3b5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:46.961781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e0afed66-ebb0-4b66-bc27-29eabbf6f3b5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.199520 May 25 16:17:46.974003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 21/41] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:46 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:47.461101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-204151d2-60b3-4428-9af4-033d1f04d4a7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:47.462477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-204151d2-60b3-4428-9af4-033d1f04d4a7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:47.475699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-204151d2-60b3-4428-9af4-033d1f04d4a7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:47.476318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-204151d2-60b3-4428-9af4-033d1f04d4a7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:47.477402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-204151d2-60b3-4428-9af4-033d1f04d4a7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:47.477993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-204151d2-60b3-4428-9af4-033d1f04d4a7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:47.509990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4a044b98-e976-40e2-a9c6-542bc2ae4506 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:47.511762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4a044b98-e976-40e2-a9c6-542bc2ae4506 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:47.520900 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a044b98-e976-40e2-a9c6-542bc2ae4506 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:47.522278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a044b98-e976-40e2-a9c6-542bc2ae4506 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:47.523285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a044b98-e976-40e2-a9c6-542bc2ae4506 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:47.524097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a044b98-e976-40e2-a9c6-542bc2ae4506 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:48.248383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-204151d2-60b3-4428-9af4-033d1f04d4a7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:48.269719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-204151d2-60b3-4428-9af4-033d1f04d4a7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.812699 May 25 16:17:48.272144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 21/42] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:47 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 816 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:48.281694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:48.283254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:48.300759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4a044b98-e976-40e2-a9c6-542bc2ae4506 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:48.308751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:48.309198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4a044b98-e976-40e2-a9c6-542bc2ae4506 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.807478 May 25 16:17:48.309783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 22/43] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:47 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 809 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:48.310054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:48.311413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:48.312051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:48.336197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-46bc8e9e-0401-4724-9b4c-e93c22b1d196 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:48.336862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-46bc8e9e-0401-4724-9b4c-e93c22b1d196 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:48.346528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-46bc8e9e-0401-4724-9b4c-e93c22b1d196 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:48.346981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-46bc8e9e-0401-4724-9b4c-e93c22b1d196 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:48.347837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-46bc8e9e-0401-4724-9b4c-e93c22b1d196 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:48.348269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-46bc8e9e-0401-4724-9b4c-e93c22b1d196 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:48.501472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:48.513431 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:48.515125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:48.527239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-862b8036-12fc-45e8-b1a3-308179608dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1339 microversion: 2.20 time: 0.242150 May 25 16:17:48.535575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 22/44] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1339 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:48.535575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-75f23d1b-94c6-4984-84ae-6fb756ed3921 req-37de52fc-d1bc-4bce-827a-5ba15181b098 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5ce685f-a7e1-41d6-96f4-e5cc19cad00a", "tag": "bb6d609e-7a8d-4258-adc8-66a6a5a24851"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:48.553448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-75f23d1b-94c6-4984-84ae-6fb756ed3921 req-37de52fc-d1bc-4bce-827a-5ba15181b098 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:48.553448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-75f23d1b-94c6-4984-84ae-6fb756ed3921 req-37de52fc-d1bc-4bce-827a-5ba15181b098 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:48.576209 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-75f23d1b-94c6-4984-84ae-6fb756ed3921 req-37de52fc-d1bc-4bce-827a-5ba15181b098 service nova] Creating event network-changed:bb6d609e-7a8d-4258-adc8-66a6a5a24851 for instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a on hv-cinder-84327 May 25 16:17:48.591279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-75f23d1b-94c6-4984-84ae-6fb756ed3921 req-37de52fc-d1bc-4bce-827a-5ba15181b098 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061572 May 25 16:17:48.592167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 23/45] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:17:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:17:48.598733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-501b52d2-4484-4ab2-84a1-1a706e49d01a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:48.599656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-501b52d2-4484-4ab2-84a1-1a706e49d01a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:48.606784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-501b52d2-4484-4ab2-84a1-1a706e49d01a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:48.607488 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-501b52d2-4484-4ab2-84a1-1a706e49d01a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:48.608873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-501b52d2-4484-4ab2-84a1-1a706e49d01a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:48.609453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-501b52d2-4484-4ab2-84a1-1a706e49d01a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:48.614956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-46bc8e9e-0401-4724-9b4c-e93c22b1d196 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:48.627642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-46bc8e9e-0401-4724-9b4c-e93c22b1d196 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1299 microversion: 2.1 time: 0.290933 May 25 16:17:48.637826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 23/46] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:48 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1299 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:49.050910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-501b52d2-4484-4ab2-84a1-1a706e49d01a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:49.060193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-501b52d2-4484-4ab2-84a1-1a706e49d01a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.466702 May 25 16:17:49.061544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 24/47] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:48 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:49.288041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-24026e24-bb86-4490-a514-91c4053f88ed tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:49.288879 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-24026e24-bb86-4490-a514-91c4053f88ed tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:49.301184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24026e24-bb86-4490-a514-91c4053f88ed tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:49.302473 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24026e24-bb86-4490-a514-91c4053f88ed tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:49.303907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24026e24-bb86-4490-a514-91c4053f88ed tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:49.305198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24026e24-bb86-4490-a514-91c4053f88ed tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:49.325704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-68470233-0696-47f5-96f0-4718dd702efe tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:49.326313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-68470233-0696-47f5-96f0-4718dd702efe tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:49.333160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-68470233-0696-47f5-96f0-4718dd702efe tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:49.333894 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-68470233-0696-47f5-96f0-4718dd702efe tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:49.334807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-68470233-0696-47f5-96f0-4718dd702efe tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:49.335502 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-68470233-0696-47f5-96f0-4718dd702efe tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:49.762040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-24026e24-bb86-4490-a514-91c4053f88ed tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:49.788368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-24026e24-bb86-4490-a514-91c4053f88ed tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.504293 May 25 16:17:49.789323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 24/48] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:49 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:49.810320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:49.811703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:49.815482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-68470233-0696-47f5-96f0-4718dd702efe tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:49.819389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:49.822059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:49.822059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:49.822059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:49.848001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-68470233-0696-47f5-96f0-4718dd702efe tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.525505 May 25 16:17:49.850951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 25/49] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:49 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:49.862090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-47aed70c-3bf4-484d-a87f-7368f942ba4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:49.865040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-47aed70c-3bf4-484d-a87f-7368f942ba4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:49.879579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47aed70c-3bf4-484d-a87f-7368f942ba4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:49.880043 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47aed70c-3bf4-484d-a87f-7368f942ba4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:49.880804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47aed70c-3bf4-484d-a87f-7368f942ba4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:49.889618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47aed70c-3bf4-484d-a87f-7368f942ba4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:50.145194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:50.153767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:50.155365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-47aed70c-3bf4-484d-a87f-7368f942ba4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:50.155510 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:50.156670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1d0ea8ae-7604-4792-b748-ae3eedc82efd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.366775 May 25 16:17:50.157315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 25/50] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:50.167301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-47aed70c-3bf4-484d-a87f-7368f942ba4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.312082 May 25 16:17:50.167301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 26/51] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:49 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:50.172075 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9cbf88a1-f3df-4bfd-ac4c-0abcdfb62e5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:50.172887 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9cbf88a1-f3df-4bfd-ac4c-0abcdfb62e5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:50.178992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cbf88a1-f3df-4bfd-ac4c-0abcdfb62e5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:50.179626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cbf88a1-f3df-4bfd-ac4c-0abcdfb62e5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:50.180457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cbf88a1-f3df-4bfd-ac4c-0abcdfb62e5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:50.180959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cbf88a1-f3df-4bfd-ac4c-0abcdfb62e5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:50.381116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9cbf88a1-f3df-4bfd-ac4c-0abcdfb62e5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:50.389083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9cbf88a1-f3df-4bfd-ac4c-0abcdfb62e5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.223281 May 25 16:17:50.389652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 27/52] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:50 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:50.805351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b755a674-6358-4337-9991-7e4bff554df3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:50.806025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b755a674-6358-4337-9991-7e4bff554df3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:50.819566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b755a674-6358-4337-9991-7e4bff554df3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:50.819946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b755a674-6358-4337-9991-7e4bff554df3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:50.820716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b755a674-6358-4337-9991-7e4bff554df3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:50.821150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b755a674-6358-4337-9991-7e4bff554df3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:50.868974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-59200686-d4ae-4c4b-b3cd-1e1039a38c1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:50.869784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-59200686-d4ae-4c4b-b3cd-1e1039a38c1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:50.881171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-59200686-d4ae-4c4b-b3cd-1e1039a38c1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:50.881635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-59200686-d4ae-4c4b-b3cd-1e1039a38c1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:50.882443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-59200686-d4ae-4c4b-b3cd-1e1039a38c1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:50.882878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-59200686-d4ae-4c4b-b3cd-1e1039a38c1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:51.030710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b755a674-6358-4337-9991-7e4bff554df3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:51.039192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b755a674-6358-4337-9991-7e4bff554df3 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.237666 May 25 16:17:51.040091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 26/53] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:50 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:51.061542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-59200686-d4ae-4c4b-b3cd-1e1039a38c1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:51.069331 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-59200686-d4ae-4c4b-b3cd-1e1039a38c1f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.208442 May 25 16:17:51.069910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 28/54] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:50 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:51.175489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:51.176597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:51.182056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-05320ebc-d68d-4e48-b5e9-5dc46620481e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:51.182609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-05320ebc-d68d-4e48-b5e9-5dc46620481e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:51.184478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:51.185495 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:51.186913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:51.188794 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:51.190349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-05320ebc-d68d-4e48-b5e9-5dc46620481e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:51.191088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-05320ebc-d68d-4e48-b5e9-5dc46620481e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:51.191900 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-05320ebc-d68d-4e48-b5e9-5dc46620481e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:51.192514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-05320ebc-d68d-4e48-b5e9-5dc46620481e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:51.211848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:51.403510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-05320ebc-d68d-4e48-b5e9-5dc46620481e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:51.405655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:51.412204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-05320ebc-d68d-4e48-b5e9-5dc46620481e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.234526 May 25 16:17:51.421871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 29/55] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:51 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:51.422355 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:51.422355 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:51.422535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7c054f29-ab1a-4efb-bee5-cfee641f5e45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:51.422618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dd044f82-9e21-45fd-a472-0907ea7a1bdb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.246836 May 25 16:17:51.422743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7c054f29-ab1a-4efb-bee5-cfee641f5e45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:51.429090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 27/56] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:51.433756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c054f29-ab1a-4efb-bee5-cfee641f5e45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:51.434368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c054f29-ab1a-4efb-bee5-cfee641f5e45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:51.435377 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c054f29-ab1a-4efb-bee5-cfee641f5e45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:51.435870 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c054f29-ab1a-4efb-bee5-cfee641f5e45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:51.586509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7c054f29-ab1a-4efb-bee5-cfee641f5e45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:51.594435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7c054f29-ab1a-4efb-bee5-cfee641f5e45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.180179 May 25 16:17:51.595003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 30/57] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:51 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:52.058375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3742d7f5-b989-448b-ab92-6ca597ca9a4e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:52.061290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3742d7f5-b989-448b-ab92-6ca597ca9a4e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:52.071072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=20 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:52.073258 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3742d7f5-b989-448b-ab92-6ca597ca9a4e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:52.078894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3742d7f5-b989-448b-ab92-6ca597ca9a4e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:52.078894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3742d7f5-b989-448b-ab92-6ca597ca9a4e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:52.078894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3742d7f5-b989-448b-ab92-6ca597ca9a4e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:52.090028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0cfe392c-0a53-4894-b1cf-9898bdd3c1ac tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:52.091231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0cfe392c-0a53-4894-b1cf-9898bdd3c1ac tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:52.097924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cfe392c-0a53-4894-b1cf-9898bdd3c1ac tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:52.098578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cfe392c-0a53-4894-b1cf-9898bdd3c1ac tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:52.099639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cfe392c-0a53-4894-b1cf-9898bdd3c1ac tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:52.100263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cfe392c-0a53-4894-b1cf-9898bdd3c1ac tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:52.211365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=6 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:52.358082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3742d7f5-b989-448b-ab92-6ca597ca9a4e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:52.375900 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3742d7f5-b989-448b-ab92-6ca597ca9a4e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.316083 May 25 16:17:52.378275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 28/58] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:52 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:52.429147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-89a1bcdf-2010-4e8d-88c3-7e3201f037aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:52.431576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-89a1bcdf-2010-4e8d-88c3-7e3201f037aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:52.438930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0cfe392c-0a53-4894-b1cf-9898bdd3c1ac tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:52.447631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0cfe392c-0a53-4894-b1cf-9898bdd3c1ac tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.363390 May 25 16:17:52.453633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 31/59] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:52 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:52.455762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89a1bcdf-2010-4e8d-88c3-7e3201f037aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:52.456296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89a1bcdf-2010-4e8d-88c3-7e3201f037aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:52.457445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89a1bcdf-2010-4e8d-88c3-7e3201f037aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:52.457936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89a1bcdf-2010-4e8d-88c3-7e3201f037aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:52.465543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:52.468483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:52.482305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:52.485449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:52.486911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:52.488917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:53.046159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-89a1bcdf-2010-4e8d-88c3-7e3201f037aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:53.055208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-89a1bcdf-2010-4e8d-88c3-7e3201f037aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.630268 May 25 16:17:53.056433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 29/60] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:52 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:17:53.076054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6ef179bb-c391-4122-bc0b-196ce9d700a0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:53.077141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6ef179bb-c391-4122-bc0b-196ce9d700a0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:53.085084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ef179bb-c391-4122-bc0b-196ce9d700a0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:53.085594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ef179bb-c391-4122-bc0b-196ce9d700a0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:53.086375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ef179bb-c391-4122-bc0b-196ce9d700a0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:53.087032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ef179bb-c391-4122-bc0b-196ce9d700a0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:53.097916 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:53.108680 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:53.110928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:53.112745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f8f4a26b-9ea6-47c1-8cca-6113daf912d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.657490 May 25 16:17:53.113667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 32/61] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:53.334582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=8 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:53.393775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9803e35e-3bd4-4bdb-8fb3-6ebf19453e9f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:53.394935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9803e35e-3bd4-4bdb-8fb3-6ebf19453e9f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:53.397069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=24 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:53.403008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9803e35e-3bd4-4bdb-8fb3-6ebf19453e9f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:53.403486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9803e35e-3bd4-4bdb-8fb3-6ebf19453e9f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:53.404256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9803e35e-3bd4-4bdb-8fb3-6ebf19453e9f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:53.406162 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9803e35e-3bd4-4bdb-8fb3-6ebf19453e9f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:53.754835 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6ef179bb-c391-4122-bc0b-196ce9d700a0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:53.767445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6ef179bb-c391-4122-bc0b-196ce9d700a0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.709659 May 25 16:17:53.768375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 30/62] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:53 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 711 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:53.774435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-536ed9e1-807b-4e11-af73-f7b26a2d09f1 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:53.775074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-536ed9e1-807b-4e11-af73-f7b26a2d09f1 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:53.782551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-536ed9e1-807b-4e11-af73-f7b26a2d09f1 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:53.783268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-536ed9e1-807b-4e11-af73-f7b26a2d09f1 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:53.784292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-536ed9e1-807b-4e11-af73-f7b26a2d09f1 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:53.784906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-536ed9e1-807b-4e11-af73-f7b26a2d09f1 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:54.087750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9803e35e-3bd4-4bdb-8fb3-6ebf19453e9f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:54.098050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9803e35e-3bd4-4bdb-8fb3-6ebf19453e9f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.708025 May 25 16:17:54.098637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 33/63] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:53 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:17:54.110808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b91cc571-a0c1-4842-a8be-2b9c7a6dd9f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:54.111473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b91cc571-a0c1-4842-a8be-2b9c7a6dd9f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:54.127619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b91cc571-a0c1-4842-a8be-2b9c7a6dd9f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:54.128674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b91cc571-a0c1-4842-a8be-2b9c7a6dd9f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:54.129938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b91cc571-a0c1-4842-a8be-2b9c7a6dd9f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:54.130804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b91cc571-a0c1-4842-a8be-2b9c7a6dd9f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:54.342359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-536ed9e1-807b-4e11-af73-f7b26a2d09f1 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:54.353707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-536ed9e1-807b-4e11-af73-f7b26a2d09f1 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1315 microversion: 2.1 time: 0.583755 May 25 16:17:54.354449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 31/64] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:53 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1315 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:54.370661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:54.372277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:54.392012 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:54.394057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:54.397597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:54.398279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:54.705253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b91cc571-a0c1-4842-a8be-2b9c7a6dd9f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:54.723516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b91cc571-a0c1-4842-a8be-2b9c7a6dd9f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.623783 May 25 16:17:54.727939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 34/65] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:54 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:54.733305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-d4bc62e9-76fc-4d2e-9b22-30c9c546eeb3 req-23cf760f-991f-4598-89e2-17f9caec48e7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4", "tag": "4d5ed500-2623-48b8-bc81-a72eb4ffeb2d"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:54.746592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-d4bc62e9-76fc-4d2e-9b22-30c9c546eeb3 req-23cf760f-991f-4598-89e2-17f9caec48e7 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:54.747242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-d4bc62e9-76fc-4d2e-9b22-30c9c546eeb3 req-23cf760f-991f-4598-89e2-17f9caec48e7 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:54.762089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-d4bc62e9-76fc-4d2e-9b22-30c9c546eeb3 req-23cf760f-991f-4598-89e2-17f9caec48e7 service nova] Creating event network-changed:4d5ed500-2623-48b8-bc81-a72eb4ffeb2d for instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 on hv-cinder-84327 May 25 16:17:54.768281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-d4bc62e9-76fc-4d2e-9b22-30c9c546eeb3 req-23cf760f-991f-4598-89e2-17f9caec48e7 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040892 May 25 16:17:54.769228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 35/66] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:17:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:17:54.785637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a88d2adf-0118-4f22-8c92-b810e96f662c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:54.787341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a88d2adf-0118-4f22-8c92-b810e96f662c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:54.794428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a88d2adf-0118-4f22-8c92-b810e96f662c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:54.795049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a88d2adf-0118-4f22-8c92-b810e96f662c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:54.796090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a88d2adf-0118-4f22-8c92-b810e96f662c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:54.796525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a88d2adf-0118-4f22-8c92-b810e96f662c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:54.801375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:55.334146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:55.344050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:55.347362 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:55.350292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-afe9df88-8134-4eab-a5ad-b90062940212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.993809 May 25 16:17:55.362390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 32/67] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 997 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:55.370208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b57d34ad-2815-4161-aa36-596348697e3c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:55.374695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b57d34ad-2815-4161-aa36-596348697e3c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:55.385297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b57d34ad-2815-4161-aa36-596348697e3c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:55.386398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b57d34ad-2815-4161-aa36-596348697e3c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:55.387930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b57d34ad-2815-4161-aa36-596348697e3c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:55.388750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b57d34ad-2815-4161-aa36-596348697e3c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:55.565292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a88d2adf-0118-4f22-8c92-b810e96f662c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:55.584381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a88d2adf-0118-4f22-8c92-b810e96f662c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.802982 May 25 16:17:55.585398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 36/68] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:54 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 805 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:17:55.593276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ee555b98-1db5-417c-9fc6-14a550e1c464 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:55.598822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ee555b98-1db5-417c-9fc6-14a550e1c464 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:55.627910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee555b98-1db5-417c-9fc6-14a550e1c464 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:55.628848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee555b98-1db5-417c-9fc6-14a550e1c464 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:55.630310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee555b98-1db5-417c-9fc6-14a550e1c464 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:55.631529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee555b98-1db5-417c-9fc6-14a550e1c464 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:55.778660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b57d34ad-2815-4161-aa36-596348697e3c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:55.790834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b57d34ad-2815-4161-aa36-596348697e3c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.426762 May 25 16:17:55.792747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 33/69] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:55 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:55.805179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5fbf0886-0ae1-42f4-bd1b-8c47c1575ff4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:55.806146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5fbf0886-0ae1-42f4-bd1b-8c47c1575ff4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:55.820650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5fbf0886-0ae1-42f4-bd1b-8c47c1575ff4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:55.821380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5fbf0886-0ae1-42f4-bd1b-8c47c1575ff4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:55.823785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5fbf0886-0ae1-42f4-bd1b-8c47c1575ff4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:55.824643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5fbf0886-0ae1-42f4-bd1b-8c47c1575ff4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:55.997041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ee555b98-1db5-417c-9fc6-14a550e1c464 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:56.014642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ee555b98-1db5-417c-9fc6-14a550e1c464 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1501 microversion: 2.1 time: 0.424522 May 25 16:17:56.015877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 37/70] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:55 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1501 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:56.102489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5fbf0886-0ae1-42f4-bd1b-8c47c1575ff4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:56.115352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5fbf0886-0ae1-42f4-bd1b-8c47c1575ff4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.318428 May 25 16:17:56.120019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 34/71] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:55 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:56.371206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:56.371557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:56.382942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:56.383435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:56.385915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:56.387624 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:56.600231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-68341bbc-9098-48b1-8740-2c299148fdb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:56.601615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-68341bbc-9098-48b1-8740-2c299148fdb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:56.609788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68341bbc-9098-48b1-8740-2c299148fdb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:56.610252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68341bbc-9098-48b1-8740-2c299148fdb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:56.613845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68341bbc-9098-48b1-8740-2c299148fdb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:56.615084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68341bbc-9098-48b1-8740-2c299148fdb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:56.895997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:56.909810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:56.914293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:56.915732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-21fc5c78-752c-41e3-8422-5946c922c247 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.549848 May 25 16:17:56.916621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 38/72] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:56 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:56.929397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-0718e0af-e0cd-4c17-b927-a36ef83c2204 req-98dfcf0b-c1c7-4182-9842-1b81900df50f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48341208-161c-454c-b88e-56a30f695cb7", "tag": "e2ff0c6b-df5c-4cfe-b8f2-13db43352e9b"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:56.938666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-0718e0af-e0cd-4c17-b927-a36ef83c2204 req-98dfcf0b-c1c7-4182-9842-1b81900df50f service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:56.939255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-0718e0af-e0cd-4c17-b927-a36ef83c2204 req-98dfcf0b-c1c7-4182-9842-1b81900df50f service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:56.953670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-0718e0af-e0cd-4c17-b927-a36ef83c2204 req-98dfcf0b-c1c7-4182-9842-1b81900df50f service nova] Creating event network-changed:e2ff0c6b-df5c-4cfe-b8f2-13db43352e9b for instance 48341208-161c-454c-b88e-56a30f695cb7 on hv-cinder-84327 May 25 16:17:56.959052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-0718e0af-e0cd-4c17-b927-a36ef83c2204 req-98dfcf0b-c1c7-4182-9842-1b81900df50f service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039920 May 25 16:17:56.960106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 39/73] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:17:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:17:56.965784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9789860e-fcaa-4c04-b841-8aaa9c815f4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:56.966744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9789860e-fcaa-4c04-b841-8aaa9c815f4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:56.973668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9789860e-fcaa-4c04-b841-8aaa9c815f4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:56.974171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9789860e-fcaa-4c04-b841-8aaa9c815f4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:56.975144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9789860e-fcaa-4c04-b841-8aaa9c815f4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:56.976064 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9789860e-fcaa-4c04-b841-8aaa9c815f4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:57.056335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-68341bbc-9098-48b1-8740-2c299148fdb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:57.066652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-68341bbc-9098-48b1-8740-2c299148fdb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.470398 May 25 16:17:57.079552 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 35/74] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:56 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:57.142005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d0dbe338-5270-4a37-b645-102fe6eec2eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:57.142898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d0dbe338-5270-4a37-b645-102fe6eec2eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:57.150898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0dbe338-5270-4a37-b645-102fe6eec2eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:57.151536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0dbe338-5270-4a37-b645-102fe6eec2eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:57.152454 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0dbe338-5270-4a37-b645-102fe6eec2eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:57.153167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0dbe338-5270-4a37-b645-102fe6eec2eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:57.300086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9789860e-fcaa-4c04-b841-8aaa9c815f4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:57.309705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9789860e-fcaa-4c04-b841-8aaa9c815f4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.348096 May 25 16:17:57.310473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 40/75] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:56 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:57.354842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d0dbe338-5270-4a37-b645-102fe6eec2eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:57.363288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d0dbe338-5270-4a37-b645-102fe6eec2eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.227432 May 25 16:17:57.367207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 36/76] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:57 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:57.705353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0d377cbf-9ce4-4788-b1de-3f075835df9e tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3f3fde2e-01e7-469c-a53f-65c992df0b49"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:57.706513 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0d377cbf-9ce4-4788-b1de-3f075835df9e tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:57.713290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d377cbf-9ce4-4788-b1de-3f075835df9e tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:57.713989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d377cbf-9ce4-4788-b1de-3f075835df9e tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:57.714769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d377cbf-9ce4-4788-b1de-3f075835df9e tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:57.715239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d377cbf-9ce4-4788-b1de-3f075835df9e tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:57.848948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-c00b7bf3-eb06-4b2b-b3bc-3c033c19d43a req-f26e4aeb-4b92-4c90-8a2d-eb39eb0acbec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5ce685f-a7e1-41d6-96f4-e5cc19cad00a", "name": "network-vif-plugged", "status": "completed", "tag": "bb6d609e-7a8d-4258-adc8-66a6a5a24851"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:17:57.860100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-c00b7bf3-eb06-4b2b-b3bc-3c033c19d43a req-f26e4aeb-4b92-4c90-8a2d-eb39eb0acbec service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:57.861159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-c00b7bf3-eb06-4b2b-b3bc-3c033c19d43a req-f26e4aeb-4b92-4c90-8a2d-eb39eb0acbec service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:57.877017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-c00b7bf3-eb06-4b2b-b3bc-3c033c19d43a req-f26e4aeb-4b92-4c90-8a2d-eb39eb0acbec service nova] Creating event network-vif-plugged:bb6d609e-7a8d-4258-adc8-66a6a5a24851 for instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a on hv-cinder-84327 May 25 16:17:57.884559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-c00b7bf3-eb06-4b2b-b3bc-3c033c19d43a req-f26e4aeb-4b92-4c90-8a2d-eb39eb0acbec service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039821 May 25 16:17:57.885824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 37/77] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:17:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:17:57.936816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:57.937512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:57.957817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:57.958262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:57.959030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:57.959486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:58.227197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0d377cbf-9ce4-4788-b1de-3f075835df9e tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "POST /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.524322 May 25 16:17:58.229760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 41/78] 10.222.0.45 () {60 vars in 1322 bytes} [Wed May 25 16:17:57 2022] POST /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2/os-volume_attachments => generated 194 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:17:58.261537 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1a465b9b-5c6a-4f7f-8d40-eed4c959bb4c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:58.263479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1a465b9b-5c6a-4f7f-8d40-eed4c959bb4c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:58.272454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a465b9b-5c6a-4f7f-8d40-eed4c959bb4c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:58.273404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a465b9b-5c6a-4f7f-8d40-eed4c959bb4c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:58.274380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a465b9b-5c6a-4f7f-8d40-eed4c959bb4c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:58.274974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a465b9b-5c6a-4f7f-8d40-eed4c959bb4c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:58.556957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:58.566320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:58.567967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:58.569204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4edeb79b-13ee-4cc9-9ac1-69cd0a766f4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.636707 May 25 16:17:58.569999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 38/79] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:58.576792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2a728f34-e0ab-4533-b785-fae4a2516223 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:58.577633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2a728f34-e0ab-4533-b785-fae4a2516223 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:58.584663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a728f34-e0ab-4533-b785-fae4a2516223 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:58.585300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a728f34-e0ab-4533-b785-fae4a2516223 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:58.586079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a728f34-e0ab-4533-b785-fae4a2516223 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:58.586518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a728f34-e0ab-4533-b785-fae4a2516223 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:58.597375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1a465b9b-5c6a-4f7f-8d40-eed4c959bb4c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:58.605241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1a465b9b-5c6a-4f7f-8d40-eed4c959bb4c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.373494 May 25 16:17:58.605829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 42/80] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:58 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:58.610918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9c48d50d-571b-4105-bdd6-c9961781f67e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:58.611764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9c48d50d-571b-4105-bdd6-c9961781f67e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:58.618091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c48d50d-571b-4105-bdd6-c9961781f67e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:58.618545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c48d50d-571b-4105-bdd6-c9961781f67e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:58.619337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c48d50d-571b-4105-bdd6-c9961781f67e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:58.619768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c48d50d-571b-4105-bdd6-c9961781f67e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:58.782516 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2a728f34-e0ab-4533-b785-fae4a2516223 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:58.792490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2a728f34-e0ab-4533-b785-fae4a2516223 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.219779 May 25 16:17:58.793423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 39/81] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:58 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:58.811237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9c48d50d-571b-4105-bdd6-c9961781f67e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:58.819263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9c48d50d-571b-4105-bdd6-c9961781f67e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.212212 May 25 16:17:58.826464 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 43/82] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:58 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:59.587617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:59.588190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:59.594109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:59.594543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:59.595325 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:59.595761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:59.622988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e93aee86-d1fa-432a-8c46-e2432ffa1120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:59.623678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e93aee86-d1fa-432a-8c46-e2432ffa1120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:59.630882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e93aee86-d1fa-432a-8c46-e2432ffa1120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:59.631403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e93aee86-d1fa-432a-8c46-e2432ffa1120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:59.632227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e93aee86-d1fa-432a-8c46-e2432ffa1120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:59.632667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e93aee86-d1fa-432a-8c46-e2432ffa1120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:59.775914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:59.793455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:59.794958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:59.796103 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-94235300-a962-442f-985d-eb2398138d3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.212935 May 25 16:17:59.796651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 40/83] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:17:59 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:17:59.804203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e93aee86-d1fa-432a-8c46-e2432ffa1120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:59.809637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0d7f06b2-dc9f-42fb-8606-77418d96ef4c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:59.810239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0d7f06b2-dc9f-42fb-8606-77418d96ef4c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:59.812890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e93aee86-d1fa-432a-8c46-e2432ffa1120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.194753 May 25 16:17:59.813454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 44/84] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:59 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:17:59.816337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d7f06b2-dc9f-42fb-8606-77418d96ef4c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:59.816789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d7f06b2-dc9f-42fb-8606-77418d96ef4c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:59.817546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d7f06b2-dc9f-42fb-8606-77418d96ef4c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:59.817976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d7f06b2-dc9f-42fb-8606-77418d96ef4c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:59.836446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b39e061a-21a2-4378-936c-9feaaca828dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:17:59.837141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b39e061a-21a2-4378-936c-9feaaca828dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:17:59.843744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b39e061a-21a2-4378-936c-9feaaca828dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:59.844267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b39e061a-21a2-4378-936c-9feaaca828dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:59.845169 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b39e061a-21a2-4378-936c-9feaaca828dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:17:59.845703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b39e061a-21a2-4378-936c-9feaaca828dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:17:59.968060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0d7f06b2-dc9f-42fb-8606-77418d96ef4c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:17:59.980167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0d7f06b2-dc9f-42fb-8606-77418d96ef4c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.174187 May 25 16:17:59.993498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 41/85] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:59 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:00.015538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-93585386-b447-4b4f-b3b9-0387c29a8ab5 req-baa32e8a-4408-4f82-b388-a99b749f72bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69fc8c5b-82f0-4a03-8dab-381fdee5884d", "name": "network-vif-plugged", "status": "completed", "tag": "4220f323-95a7-4c14-8c3e-42c4591d25b3"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:00.039511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-93585386-b447-4b4f-b3b9-0387c29a8ab5 req-baa32e8a-4408-4f82-b388-a99b749f72bf service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:00.040334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-93585386-b447-4b4f-b3b9-0387c29a8ab5 req-baa32e8a-4408-4f82-b388-a99b749f72bf service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:00.054273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-93585386-b447-4b4f-b3b9-0387c29a8ab5 req-baa32e8a-4408-4f82-b388-a99b749f72bf service nova] Creating event network-vif-plugged:4220f323-95a7-4c14-8c3e-42c4591d25b3 for instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d on hv-cinder-84327 May 25 16:18:00.071295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-93585386-b447-4b4f-b3b9-0387c29a8ab5 req-baa32e8a-4408-4f82-b388-a99b749f72bf service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065046 May 25 16:18:00.071936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 42/86] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:00.152237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b39e061a-21a2-4378-936c-9feaaca828dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:00.159953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b39e061a-21a2-4378-936c-9feaaca828dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.327404 May 25 16:18:00.160530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 45/87] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:17:59 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:00.814377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:00.815042 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:00.824635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:00.824635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:00.831672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b527849e-3211-41fc-a355-6564aa2822cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:00.833300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b527849e-3211-41fc-a355-6564aa2822cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:00.840086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:00.840778 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:00.846091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b527849e-3211-41fc-a355-6564aa2822cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:00.848103 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b527849e-3211-41fc-a355-6564aa2822cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:00.850861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b527849e-3211-41fc-a355-6564aa2822cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:00.851881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b527849e-3211-41fc-a355-6564aa2822cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:01.088951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:01.091176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b527849e-3211-41fc-a355-6564aa2822cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:01.099262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b527849e-3211-41fc-a355-6564aa2822cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.272758 May 25 16:18:01.099872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 46/88] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:00 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:01.109385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:01.111848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:01.112526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-68ef1753-ebad-4076-9b74-82c12339fde5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:01.113146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-68ef1753-ebad-4076-9b74-82c12339fde5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:01.114026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-efcb2e92-725a-421b-aaec-3383dde22f60 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.303639 May 25 16:18:01.115296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 43/89] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:00 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:01.119381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-68ef1753-ebad-4076-9b74-82c12339fde5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:01.119995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-68ef1753-ebad-4076-9b74-82c12339fde5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:01.120945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-68ef1753-ebad-4076-9b74-82c12339fde5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:01.121526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-68ef1753-ebad-4076-9b74-82c12339fde5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:01.177785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-da524627-c786-4c2b-ad3d-988d9f56ce4e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:01.178423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-da524627-c786-4c2b-ad3d-988d9f56ce4e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:01.185499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da524627-c786-4c2b-ad3d-988d9f56ce4e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:01.186249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da524627-c786-4c2b-ad3d-988d9f56ce4e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:01.187234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da524627-c786-4c2b-ad3d-988d9f56ce4e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:01.187786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da524627-c786-4c2b-ad3d-988d9f56ce4e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:01.333874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-68ef1753-ebad-4076-9b74-82c12339fde5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:01.343788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-68ef1753-ebad-4076-9b74-82c12339fde5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.241355 May 25 16:18:01.345634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 47/90] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:01 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:01.364806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-da524627-c786-4c2b-ad3d-988d9f56ce4e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:01.372326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-da524627-c786-4c2b-ad3d-988d9f56ce4e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.199334 May 25 16:18:01.372897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 44/91] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:01 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:02.117351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-17efb756-db17-491a-b0fe-a4fa06012a0c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:02.118003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-17efb756-db17-491a-b0fe-a4fa06012a0c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:02.132463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:02.133579 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:02.134231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17efb756-db17-491a-b0fe-a4fa06012a0c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:02.135275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17efb756-db17-491a-b0fe-a4fa06012a0c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:02.136373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:02.137478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17efb756-db17-491a-b0fe-a4fa06012a0c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:02.147558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:02.148142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:02.149447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:02.150095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:02.155006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17efb756-db17-491a-b0fe-a4fa06012a0c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:02.221906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:02.582061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:02.596745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:02.599940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:02.604829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-195513e6-f551-4b5e-a720-5d40c1f1bc00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.476610 May 25 16:18:02.608177 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 45/92] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:02 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:02.620354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-248e81fa-3850-4daa-b753-1cf559efe914 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:02.621747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-248e81fa-3850-4daa-b753-1cf559efe914 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:02.625007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-17efb756-db17-491a-b0fe-a4fa06012a0c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:02.643081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-248e81fa-3850-4daa-b753-1cf559efe914 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:02.643503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-248e81fa-3850-4daa-b753-1cf559efe914 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:02.644380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-248e81fa-3850-4daa-b753-1cf559efe914 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:02.644847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-248e81fa-3850-4daa-b753-1cf559efe914 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:02.645236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-17efb756-db17-491a-b0fe-a4fa06012a0c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.531567 May 25 16:18:02.645973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 48/93] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:02 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:02.658761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7d789434-93e3-4532-aeb8-28fdaa2269da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:02.659826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7d789434-93e3-4532-aeb8-28fdaa2269da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:02.668662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d789434-93e3-4532-aeb8-28fdaa2269da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:02.669064 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d789434-93e3-4532-aeb8-28fdaa2269da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:02.670873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d789434-93e3-4532-aeb8-28fdaa2269da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:02.671482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d789434-93e3-4532-aeb8-28fdaa2269da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:02.889416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7d789434-93e3-4532-aeb8-28fdaa2269da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:02.897665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7d789434-93e3-4532-aeb8-28fdaa2269da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.250525 May 25 16:18:02.898056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-248e81fa-3850-4daa-b753-1cf559efe914 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:02.898413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 49/94] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:02 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:02.907024 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-248e81fa-3850-4daa-b753-1cf559efe914 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.295740 May 25 16:18:02.907024 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 46/95] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:02 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:03.629941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:03.631125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:03.641490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:03.642244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:03.643618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:03.644344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:03.661989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-123accca-0eb4-4647-b024-8eee88fc31c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:03.662567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-123accca-0eb4-4647-b024-8eee88fc31c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:03.669873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-123accca-0eb4-4647-b024-8eee88fc31c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:03.670330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-123accca-0eb4-4647-b024-8eee88fc31c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:03.671100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-123accca-0eb4-4647-b024-8eee88fc31c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:03.671547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-123accca-0eb4-4647-b024-8eee88fc31c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:03.821712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:03.830240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:03.831767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:03.832863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c5bc8c16-e2aa-41c4-bb59-f2c98f286cd0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.209564 May 25 16:18:03.833421 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 47/96] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:03 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:03.847667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-123accca-0eb4-4647-b024-8eee88fc31c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:03.855242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-123accca-0eb4-4647-b024-8eee88fc31c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.197066 May 25 16:18:03.855786 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 50/97] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:03 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:03.914695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-59128444-47a0-4792-89ca-ecfadab91276 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:03.915351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-59128444-47a0-4792-89ca-ecfadab91276 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:03.922054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59128444-47a0-4792-89ca-ecfadab91276 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:03.922202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6cbca651-4320-4306-8607-0b374f33f997 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:03.922507 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59128444-47a0-4792-89ca-ecfadab91276 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:03.922745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6cbca651-4320-4306-8607-0b374f33f997 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:03.923295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59128444-47a0-4792-89ca-ecfadab91276 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:03.923721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59128444-47a0-4792-89ca-ecfadab91276 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:03.929260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cbca651-4320-4306-8607-0b374f33f997 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:03.929733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cbca651-4320-4306-8607-0b374f33f997 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:03.930500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cbca651-4320-4306-8607-0b374f33f997 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:03.931041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cbca651-4320-4306-8607-0b374f33f997 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:04.115740 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-59128444-47a0-4792-89ca-ecfadab91276 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:04.118675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6cbca651-4320-4306-8607-0b374f33f997 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:04.123341 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-59128444-47a0-4792-89ca-ecfadab91276 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.212520 May 25 16:18:04.123944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 48/98] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:03 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:04.126395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6cbca651-4320-4306-8607-0b374f33f997 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.208112 May 25 16:18:04.126957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 51/99] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:03 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:04.851939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:04.852708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:04.861497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:04.862129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:04.863142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:04.863683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:04.874525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6e1caaff-7ce0-4527-b9c2-8e8384137caf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:04.875177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6e1caaff-7ce0-4527-b9c2-8e8384137caf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:04.882516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e1caaff-7ce0-4527-b9c2-8e8384137caf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:04.882976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e1caaff-7ce0-4527-b9c2-8e8384137caf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:04.883863 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e1caaff-7ce0-4527-b9c2-8e8384137caf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:04.884335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e1caaff-7ce0-4527-b9c2-8e8384137caf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:04.887119 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:05.064644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:05.069412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6e1caaff-7ce0-4527-b9c2-8e8384137caf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:05.073585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:05.075371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:05.076488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b36dee46-2348-48fe-9503-8feaabacf474 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.229964 May 25 16:18:05.077230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 49/100] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:04 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:05.078172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6e1caaff-7ce0-4527-b9c2-8e8384137caf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.207886 May 25 16:18:05.078880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 52/101] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:04 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:05.140579 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-490335d5-a7c7-4f02-974b-c843be54b830 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:05.141184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-490335d5-a7c7-4f02-974b-c843be54b830 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:05.143031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5734305e-a923-4cd4-bc71-605e1116855e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:05.143662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5734305e-a923-4cd4-bc71-605e1116855e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:05.147749 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-490335d5-a7c7-4f02-974b-c843be54b830 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:05.148196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-490335d5-a7c7-4f02-974b-c843be54b830 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:05.148960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-490335d5-a7c7-4f02-974b-c843be54b830 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:05.149395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-490335d5-a7c7-4f02-974b-c843be54b830 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:05.149944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5734305e-a923-4cd4-bc71-605e1116855e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:05.150400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5734305e-a923-4cd4-bc71-605e1116855e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:05.151175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5734305e-a923-4cd4-bc71-605e1116855e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:05.151593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5734305e-a923-4cd4-bc71-605e1116855e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:05.344375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-490335d5-a7c7-4f02-974b-c843be54b830 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:05.352100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5734305e-a923-4cd4-bc71-605e1116855e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:05.353483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-490335d5-a7c7-4f02-974b-c843be54b830 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.216781 May 25 16:18:05.354534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 50/102] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:05 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:05.367611 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5734305e-a923-4cd4-bc71-605e1116855e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1363 microversion: 2.1 time: 0.228209 May 25 16:18:05.368187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 53/103] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:05 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1363 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:06.094576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:06.095098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5942c4ff-6e67-42d1-be6f-1d12f4cce1f1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:06.095520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:06.095618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5942c4ff-6e67-42d1-be6f-1d12f4cce1f1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:06.101676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.101930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5942c4ff-6e67-42d1-be6f-1d12f4cce1f1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.102114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.102384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5942c4ff-6e67-42d1-be6f-1d12f4cce1f1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.102995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.103121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5942c4ff-6e67-42d1-be6f-1d12f4cce1f1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.103482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.103614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5942c4ff-6e67-42d1-be6f-1d12f4cce1f1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.307369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5942c4ff-6e67-42d1-be6f-1d12f4cce1f1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:06.309093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5942c4ff-6e67-42d1-be6f-1d12f4cce1f1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.217512 May 25 16:18:06.314433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:06.321969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 54/104] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:06 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:06.341471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:06.347696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:06.349174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-022676b1-f41e-454b-9828-603b6a0df337 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.258601 May 25 16:18:06.352130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 51/105] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:06 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:06.370086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5220da1e-4935-4f4c-aa15-485d40d760d2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:06.371025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5220da1e-4935-4f4c-aa15-485d40d760d2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:06.385184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5220da1e-4935-4f4c-aa15-485d40d760d2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.387682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5220da1e-4935-4f4c-aa15-485d40d760d2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.388723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5220da1e-4935-4f4c-aa15-485d40d760d2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.389383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5220da1e-4935-4f4c-aa15-485d40d760d2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.395782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d8afc83a-980c-4707-a18b-195225964657 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:06.396599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d8afc83a-980c-4707-a18b-195225964657 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:06.405809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8afc83a-980c-4707-a18b-195225964657 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.406467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8afc83a-980c-4707-a18b-195225964657 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.409692 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8afc83a-980c-4707-a18b-195225964657 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.410398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8afc83a-980c-4707-a18b-195225964657 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.672820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5220da1e-4935-4f4c-aa15-485d40d760d2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:06.675580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d8afc83a-980c-4707-a18b-195225964657 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:06.682297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5220da1e-4935-4f4c-aa15-485d40d760d2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.316283 May 25 16:18:06.683352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 55/106] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:06 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:06.689306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-a475f32e-856c-44a9-9d5f-f076447fd1a6 req-398010e4-c832-447b-9de0-7f104fb28b7e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f858e1ab-1766-4273-be90-d3c73f8678da", "name": "network-vif-plugged", "status": "completed", "tag": "dc3d60cd-b59c-445a-84d9-b0658a3a8344"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:06.693632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d8afc83a-980c-4707-a18b-195225964657 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1549 microversion: 2.1 time: 0.308861 May 25 16:18:06.694502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 52/107] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:06 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1549 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:06.699880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-a475f32e-856c-44a9-9d5f-f076447fd1a6 req-398010e4-c832-447b-9de0-7f104fb28b7e service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.700654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-a475f32e-856c-44a9-9d5f-f076447fd1a6 req-398010e4-c832-447b-9de0-7f104fb28b7e service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.714012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-a475f32e-856c-44a9-9d5f-f076447fd1a6 req-398010e4-c832-447b-9de0-7f104fb28b7e service nova] Creating event network-vif-plugged:dc3d60cd-b59c-445a-84d9-b0658a3a8344 for instance f858e1ab-1766-4273-be90-d3c73f8678da on hv-cinder-84327 May 25 16:18:06.714367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-992ede98-d9af-4e6b-9026-f4d6500135d2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:06.715014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-992ede98-d9af-4e6b-9026-f4d6500135d2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:06.720544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-a475f32e-856c-44a9-9d5f-f076447fd1a6 req-398010e4-c832-447b-9de0-7f104fb28b7e service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035699 May 25 16:18:06.721397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 56/108] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:06.724026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-992ede98-d9af-4e6b-9026-f4d6500135d2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.724578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-992ede98-d9af-4e6b-9026-f4d6500135d2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.725584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-992ede98-d9af-4e6b-9026-f4d6500135d2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:06.726396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-992ede98-d9af-4e6b-9026-f4d6500135d2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:06.852946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-992ede98-d9af-4e6b-9026-f4d6500135d2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.144963 May 25 16:18:06.859378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 53/109] 10.222.0.45 () {58 vars in 1274 bytes} [Wed May 25 16:18:06 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/os-interface => generated 285 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:07.331023 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9fb71c6a-07f2-4d53-8074-819f579c0aed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:07.332531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9fb71c6a-07f2-4d53-8074-819f579c0aed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:07.339859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9fb71c6a-07f2-4d53-8074-819f579c0aed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:07.340779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9fb71c6a-07f2-4d53-8074-819f579c0aed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:07.341744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9fb71c6a-07f2-4d53-8074-819f579c0aed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:07.342509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9fb71c6a-07f2-4d53-8074-819f579c0aed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:07.372832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:07.375006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:07.382533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:07.384876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:07.384876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:07.384876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:07.553299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9fb71c6a-07f2-4d53-8074-819f579c0aed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:07.561457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9fb71c6a-07f2-4d53-8074-819f579c0aed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.236866 May 25 16:18:07.562418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 57/110] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:07 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:07.576027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:07.584157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:07.585636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:07.586772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ebb25be8-edfa-4191-9085-e4f8cda746e9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.218682 May 25 16:18:07.587364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 54/111] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:07 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:18:07.700317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0b412e62-1686-4764-874c-6d40d1ce011e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:07.700950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0b412e62-1686-4764-874c-6d40d1ce011e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:07.710362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b412e62-1686-4764-874c-6d40d1ce011e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:07.710821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b412e62-1686-4764-874c-6d40d1ce011e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:07.711649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b412e62-1686-4764-874c-6d40d1ce011e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:07.712087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b412e62-1686-4764-874c-6d40d1ce011e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:07.955462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0b412e62-1686-4764-874c-6d40d1ce011e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:07.965518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0b412e62-1686-4764-874c-6d40d1ce011e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.269553 May 25 16:18:07.976351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 58/112] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:07 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:08.500425 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-18be2fd7-9b7d-439c-9560-c58eb13aaac0 req-087d885a-173a-42fd-acbc-49ceadb1a4a5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5ce685f-a7e1-41d6-96f4-e5cc19cad00a", "tag": "bb6d609e-7a8d-4258-adc8-66a6a5a24851"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:08.510254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-18be2fd7-9b7d-439c-9560-c58eb13aaac0 req-087d885a-173a-42fd-acbc-49ceadb1a4a5 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:08.510816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-18be2fd7-9b7d-439c-9560-c58eb13aaac0 req-087d885a-173a-42fd-acbc-49ceadb1a4a5 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:08.524353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-18be2fd7-9b7d-439c-9560-c58eb13aaac0 req-087d885a-173a-42fd-acbc-49ceadb1a4a5 service nova] Creating event network-changed:bb6d609e-7a8d-4258-adc8-66a6a5a24851 for instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a on hv-cinder-84327 May 25 16:18:08.531635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-18be2fd7-9b7d-439c-9560-c58eb13aaac0 req-087d885a-173a-42fd-acbc-49ceadb1a4a5 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036507 May 25 16:18:08.532574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 55/113] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:08.592198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5a124513-faf2-41d5-9a56-7027619ac6ef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:08.593037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5a124513-faf2-41d5-9a56-7027619ac6ef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:08.601132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5a124513-faf2-41d5-9a56-7027619ac6ef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:08.601786 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5a124513-faf2-41d5-9a56-7027619ac6ef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:08.602567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5a124513-faf2-41d5-9a56-7027619ac6ef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:08.603178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5a124513-faf2-41d5-9a56-7027619ac6ef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:08.605751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:08.606473 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:08.619349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:08.620105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:08.622190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:08.623575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:08.866778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5a124513-faf2-41d5-9a56-7027619ac6ef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:08.880259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5a124513-faf2-41d5-9a56-7027619ac6ef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.298032 May 25 16:18:08.891183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 59/114] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:08 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:08.896633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-b6a6abe3-2a66-46e5-bd96-f23e982d14e9 req-267135ee-3fca-4c67-b758-140d0d94be7a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4", "name": "network-vif-plugged", "status": "completed", "tag": "4d5ed500-2623-48b8-bc81-a72eb4ffeb2d"}, {"server_uuid": "48341208-161c-454c-b88e-56a30f695cb7", "name": "network-vif-plugged", "status": "completed", "tag": "e2ff0c6b-df5c-4cfe-b8f2-13db43352e9b"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:08.902784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:08.910938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:08.912424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:08.913545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9a8b8234-c80f-47de-b33c-f9d239af5e53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.312413 May 25 16:18:08.930383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-b6a6abe3-2a66-46e5-bd96-f23e982d14e9 req-267135ee-3fca-4c67-b758-140d0d94be7a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:08.930829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-b6a6abe3-2a66-46e5-bd96-f23e982d14e9 req-267135ee-3fca-4c67-b758-140d0d94be7a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:08.931093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 56/115] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:08 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:18:08.945767 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-b6a6abe3-2a66-46e5-bd96-f23e982d14e9 req-267135ee-3fca-4c67-b758-140d0d94be7a service nova] Creating event network-vif-plugged:4d5ed500-2623-48b8-bc81-a72eb4ffeb2d for instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 on hv-cinder-84327 May 25 16:18:08.946276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-b6a6abe3-2a66-46e5-bd96-f23e982d14e9 req-267135ee-3fca-4c67-b758-140d0d94be7a service nova] Creating event network-vif-plugged:e2ff0c6b-df5c-4cfe-b8f2-13db43352e9b for instance 48341208-161c-454c-b88e-56a30f695cb7 on hv-cinder-84327 May 25 16:18:08.953251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-b6a6abe3-2a66-46e5-bd96-f23e982d14e9 req-267135ee-3fca-4c67-b758-140d0d94be7a service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.060633 May 25 16:18:08.953953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 60/116] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:08 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:08.983123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-89897a01-62b7-4fe8-a644-87b1e4674e64 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:08.983898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-89897a01-62b7-4fe8-a644-87b1e4674e64 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:08.990596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89897a01-62b7-4fe8-a644-87b1e4674e64 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:08.991055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89897a01-62b7-4fe8-a644-87b1e4674e64 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:08.991854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89897a01-62b7-4fe8-a644-87b1e4674e64 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:08.992452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89897a01-62b7-4fe8-a644-87b1e4674e64 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:09.182130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-89897a01-62b7-4fe8-a644-87b1e4674e64 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:09.193406 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-89897a01-62b7-4fe8-a644-87b1e4674e64 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.214126 May 25 16:18:09.194364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 57/117] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:08 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:09.899128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-77d67380-bbb3-4108-95c1-54aaa6f88a0a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:09.899779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-77d67380-bbb3-4108-95c1-54aaa6f88a0a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:09.907658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77d67380-bbb3-4108-95c1-54aaa6f88a0a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:09.908113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77d67380-bbb3-4108-95c1-54aaa6f88a0a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:09.908891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77d67380-bbb3-4108-95c1-54aaa6f88a0a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:09.909398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77d67380-bbb3-4108-95c1-54aaa6f88a0a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:09.932147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:09.933439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:09.942535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:09.943227 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:09.944179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:09.944846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:10.144623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-77d67380-bbb3-4108-95c1-54aaa6f88a0a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:10.155222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-77d67380-bbb3-4108-95c1-54aaa6f88a0a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.261005 May 25 16:18:10.164306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 61/118] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:09 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:10.164799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:10.166684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:10.168227 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:10.169352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-75c5c0d5-79cf-4e89-93ae-062c78785cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.241812 May 25 16:18:10.169913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 58/119] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:09 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:18:10.210944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-239623cd-8580-4a1c-b56f-f8f5da44b876 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:10.210944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-239623cd-8580-4a1c-b56f-f8f5da44b876 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:10.239101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-239623cd-8580-4a1c-b56f-f8f5da44b876 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:10.239601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-239623cd-8580-4a1c-b56f-f8f5da44b876 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:10.240395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-239623cd-8580-4a1c-b56f-f8f5da44b876 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:10.240832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-239623cd-8580-4a1c-b56f-f8f5da44b876 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:10.421385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-239623cd-8580-4a1c-b56f-f8f5da44b876 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:10.436076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-239623cd-8580-4a1c-b56f-f8f5da44b876 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.229427 May 25 16:18:10.447581 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 62/120] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:10 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:11.172928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c88baa23-917a-4a0e-9897-c3f747c80715 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:11.173601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c88baa23-917a-4a0e-9897-c3f747c80715 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:11.180166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c88baa23-917a-4a0e-9897-c3f747c80715 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:11.188692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:11.189853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:11.203875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c88baa23-917a-4a0e-9897-c3f747c80715 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:11.207860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c88baa23-917a-4a0e-9897-c3f747c80715 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:11.208521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c88baa23-917a-4a0e-9897-c3f747c80715 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:11.212173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:11.213527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:11.214963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:11.215633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:11.448437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:11.453332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c88baa23-917a-4a0e-9897-c3f747c80715 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:11.457579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:11.459810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:11.461365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c651c60c-ee27-4d03-ac23-8658578f4029 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.277656 May 25 16:18:11.462657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c88baa23-917a-4a0e-9897-c3f747c80715 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.294355 May 25 16:18:11.472268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 63/122] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:11 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:11.472484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 59/122] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:11 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:11.477379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fd1361a2-9288-4823-8290-a738fc136d3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:11.477984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fd1361a2-9288-4823-8290-a738fc136d3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:11.484171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd1361a2-9288-4823-8290-a738fc136d3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:11.484626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd1361a2-9288-4823-8290-a738fc136d3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:11.485388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd1361a2-9288-4823-8290-a738fc136d3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:11.485817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd1361a2-9288-4823-8290-a738fc136d3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:11.624092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fd1361a2-9288-4823-8290-a738fc136d3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:11.633418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fd1361a2-9288-4823-8290-a738fc136d3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.159827 May 25 16:18:11.634188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 64/123] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:11 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:12.479399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:12.479962 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-20e46314-7bda-4ddd-a1bb-122c530f4f3b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:12.480329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:12.480406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-20e46314-7bda-4ddd-a1bb-122c530f4f3b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:12.486093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:12.486877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:12.487855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:12.488275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:12.488381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-20e46314-7bda-4ddd-a1bb-122c530f4f3b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:12.488800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-20e46314-7bda-4ddd-a1bb-122c530f4f3b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:12.489563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-20e46314-7bda-4ddd-a1bb-122c530f4f3b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:12.490000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-20e46314-7bda-4ddd-a1bb-122c530f4f3b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:12.747620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-20e46314-7bda-4ddd-a1bb-122c530f4f3b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:12.756722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:12.756884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-20e46314-7bda-4ddd-a1bb-122c530f4f3b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.280966 May 25 16:18:12.767251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 65/124] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:12 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:12.767803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:12.773058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-77c2016e-b5a9-403f-8a20-778b62c43810 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:12.773662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-77c2016e-b5a9-403f-8a20-778b62c43810 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:12.780292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77c2016e-b5a9-403f-8a20-778b62c43810 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:12.780750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77c2016e-b5a9-403f-8a20-778b62c43810 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:12.781504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77c2016e-b5a9-403f-8a20-778b62c43810 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:12.781967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77c2016e-b5a9-403f-8a20-778b62c43810 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:12.786793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:12.790525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7d4dbad2-6a5e-4f2b-9a80-72ed84f4233c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.313708 May 25 16:18:12.802385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 60/125] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:12 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:12.995250 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-77c2016e-b5a9-403f-8a20-778b62c43810 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:13.005605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-77c2016e-b5a9-403f-8a20-778b62c43810 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.235716 May 25 16:18:13.006447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 66/126] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:12 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:13.532768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cbe867b0-aedb-4302-8cd7-dbc2e750234f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:13.533299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cbe867b0-aedb-4302-8cd7-dbc2e750234f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:13.543898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cbe867b0-aedb-4302-8cd7-dbc2e750234f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:13.544525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cbe867b0-aedb-4302-8cd7-dbc2e750234f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:13.545614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cbe867b0-aedb-4302-8cd7-dbc2e750234f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:13.546230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cbe867b0-aedb-4302-8cd7-dbc2e750234f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:13.573480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cbe867b0-aedb-4302-8cd7-dbc2e750234f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Going to try to terminate instance {{(pid=117795) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:18:13.641088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:13.693644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cbe867b0-aedb-4302-8cd7-dbc2e750234f tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "DELETE /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 204 len: 0 microversion: 2.1 time: 0.166281 May 25 16:18:13.694239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 61/127] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:18:13 2022] DELETE /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:18:13.702395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aabbd75b-eecb-43b8-bf76-6b301c0e443c tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:13.703001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aabbd75b-eecb-43b8-bf76-6b301c0e443c tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:13.705690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:13.712815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aabbd75b-eecb-43b8-bf76-6b301c0e443c tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:13.713263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aabbd75b-eecb-43b8-bf76-6b301c0e443c tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:13.714018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aabbd75b-eecb-43b8-bf76-6b301c0e443c tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:13.714443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aabbd75b-eecb-43b8-bf76-6b301c0e443c tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:13.776104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fead9a57-aa6c-416f-ac01-03c84f923f1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:13.777556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fead9a57-aa6c-416f-ac01-03c84f923f1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:13.785593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fead9a57-aa6c-416f-ac01-03c84f923f1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:13.786394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fead9a57-aa6c-416f-ac01-03c84f923f1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:13.787551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fead9a57-aa6c-416f-ac01-03c84f923f1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:13.788190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fead9a57-aa6c-416f-ac01-03c84f923f1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:13.910979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aabbd75b-eecb-43b8-bf76-6b301c0e443c tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:13.920808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aabbd75b-eecb-43b8-bf76-6b301c0e443c tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1553 microversion: 2.1 time: 0.221997 May 25 16:18:13.921703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 67/128] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:13 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1553 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:13.926903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:13.927518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:13.933989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:13.934587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:13.935552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:13.935989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:13.993190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fead9a57-aa6c-416f-ac01-03c84f923f1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:14.002352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fead9a57-aa6c-416f-ac01-03c84f923f1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1322 microversion: 2.1 time: 0.231381 May 25 16:18:14.002962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 62/129] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:13 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1322 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:14.022426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7b98af29-d25b-4955-90b5-70e6f325111e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:14.023317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7b98af29-d25b-4955-90b5-70e6f325111e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:14.030120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7b98af29-d25b-4955-90b5-70e6f325111e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:14.030630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7b98af29-d25b-4955-90b5-70e6f325111e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:14.031699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7b98af29-d25b-4955-90b5-70e6f325111e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:14.032537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7b98af29-d25b-4955-90b5-70e6f325111e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:14.118528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:14.128816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:14.130866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:14.132204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2a064693-860e-4752-8a43-4f5670a0576c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.209253 May 25 16:18:14.132769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 68/130] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:13 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:14.192248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7b98af29-d25b-4955-90b5-70e6f325111e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:14.200773 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7b98af29-d25b-4955-90b5-70e6f325111e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.182592 May 25 16:18:14.201601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 63/131] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:14 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:14.937644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-36672db6-1e66-4c0b-8a6a-f1f95f3856f6 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:14.938276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-36672db6-1e66-4c0b-8a6a-f1f95f3856f6 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:14.950449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-36672db6-1e66-4c0b-8a6a-f1f95f3856f6 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:14.950898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-36672db6-1e66-4c0b-8a6a-f1f95f3856f6 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:14.951687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-36672db6-1e66-4c0b-8a6a-f1f95f3856f6 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:14.952117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-36672db6-1e66-4c0b-8a6a-f1f95f3856f6 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.019546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c52e4cfb-b74d-48ae-8848-17656a985220 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:15.020159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c52e4cfb-b74d-48ae-8848-17656a985220 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:15.029822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c52e4cfb-b74d-48ae-8848-17656a985220 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.029922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c52e4cfb-b74d-48ae-8848-17656a985220 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.031334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c52e4cfb-b74d-48ae-8848-17656a985220 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.031334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c52e4cfb-b74d-48ae-8848-17656a985220 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.180962 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-36672db6-1e66-4c0b-8a6a-f1f95f3856f6 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:15.191293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-36672db6-1e66-4c0b-8a6a-f1f95f3856f6 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1553 microversion: 2.1 time: 0.257247 May 25 16:18:15.201909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 69/132] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:14 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1553 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:15.207813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:15.208682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:15.216119 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.216808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.217797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.218452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.222918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:15.224577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c52e4cfb-b74d-48ae-8848-17656a985220 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:15.238069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c52e4cfb-b74d-48ae-8848-17656a985220 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1516 microversion: 2.1 time: 0.222668 May 25 16:18:15.238658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 64/133] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:15 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1516 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:15.247832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-389d14df-a022-47ee-9041-58be225be709 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:15.250766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-389d14df-a022-47ee-9041-58be225be709 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:15.257567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-389d14df-a022-47ee-9041-58be225be709 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.258257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-389d14df-a022-47ee-9041-58be225be709 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.259024 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-389d14df-a022-47ee-9041-58be225be709 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.259492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-389d14df-a022-47ee-9041-58be225be709 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.418232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:15.444374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:15.449955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:15.451500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3131f8fe-d454-43b5-8291-3458aa0be75e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.248252 May 25 16:18:15.452520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 70/134] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:15 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:15.456525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-389d14df-a022-47ee-9041-58be225be709 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:15.466852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-389d14df-a022-47ee-9041-58be225be709 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.226963 May 25 16:18:15.467492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 65/135] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:15 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:15.482253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ba636191-e75f-441c-9727-9d3e89da190f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:15.483855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ba636191-e75f-441c-9727-9d3e89da190f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:15.491014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ba636191-e75f-441c-9727-9d3e89da190f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.491480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ba636191-e75f-441c-9727-9d3e89da190f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.492245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ba636191-e75f-441c-9727-9d3e89da190f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.492673 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ba636191-e75f-441c-9727-9d3e89da190f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.517604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-ba636191-e75f-441c-9727-9d3e89da190f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lazy-loading 'flavor' on Instance uuid f858e1ab-1766-4273-be90-d3c73f8678da {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:18:15.648117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ba636191-e75f-441c-9727-9d3e89da190f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da/action" status: 200 len: 32 microversion: 2.1 time: 0.186001 May 25 16:18:15.649277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 71/136] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:18:15 2022] POST /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da/action => generated 32 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 25 16:18:15.659507 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-09d63da6-1abe-49f0-816f-df7358d8c146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:15.660235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-09d63da6-1abe-49f0-816f-df7358d8c146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:15.668210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09d63da6-1abe-49f0-816f-df7358d8c146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.669184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09d63da6-1abe-49f0-816f-df7358d8c146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:15.670567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09d63da6-1abe-49f0-816f-df7358d8c146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:15.671545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09d63da6-1abe-49f0-816f-df7358d8c146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:16.070364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-09d63da6-1abe-49f0-816f-df7358d8c146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:16.077784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-09d63da6-1abe-49f0-816f-df7358d8c146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.422316 May 25 16:18:16.078346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 66/137] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:15 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:16.209723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-32f450e4-e620-496e-91fa-df23cace49a2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:16.210505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-32f450e4-e620-496e-91fa-df23cace49a2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:16.220000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32f450e4-e620-496e-91fa-df23cace49a2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:16.220710 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32f450e4-e620-496e-91fa-df23cace49a2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:16.221714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32f450e4-e620-496e-91fa-df23cace49a2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:16.222165 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32f450e4-e620-496e-91fa-df23cace49a2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:16.394009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-32f450e4-e620-496e-91fa-df23cace49a2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:16.408904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-32f450e4-e620-496e-91fa-df23cace49a2 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1553 microversion: 2.1 time: 0.203096 May 25 16:18:16.412336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 72/138] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:16 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1553 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:16.469499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:16.470540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:16.479388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:16.479838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:16.481430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:16.481857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:16.482813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-adeedc79-4e12-4b12-9e80-d3312a030eaf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:16.483446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-adeedc79-4e12-4b12-9e80-d3312a030eaf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:16.490160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adeedc79-4e12-4b12-9e80-d3312a030eaf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:16.491022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adeedc79-4e12-4b12-9e80-d3312a030eaf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:16.492153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adeedc79-4e12-4b12-9e80-d3312a030eaf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:16.492805 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adeedc79-4e12-4b12-9e80-d3312a030eaf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:16.764947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:16.776144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-adeedc79-4e12-4b12-9e80-d3312a030eaf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:16.793971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-adeedc79-4e12-4b12-9e80-d3312a030eaf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1367 microversion: 2.1 time: 0.314828 May 25 16:18:16.794806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:16.795543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 73/139] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:16 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1367 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:16.806232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:16.807460 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-714ae2d4-7516-407c-87ed-1ce08b35f8ed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1407 microversion: 2.20 time: 0.341799 May 25 16:18:16.817578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 67/140] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:16 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1407 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:17.096128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cb413d1f-418f-4f3f-b646-d3a82ecb5578 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:17.096764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cb413d1f-418f-4f3f-b646-d3a82ecb5578 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:17.103631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb413d1f-418f-4f3f-b646-d3a82ecb5578 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:17.104339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb413d1f-418f-4f3f-b646-d3a82ecb5578 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:17.105336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb413d1f-418f-4f3f-b646-d3a82ecb5578 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:17.106074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb413d1f-418f-4f3f-b646-d3a82ecb5578 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:17.251370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cb413d1f-418f-4f3f-b646-d3a82ecb5578 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:17.262519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cb413d1f-418f-4f3f-b646-d3a82ecb5578 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.171429 May 25 16:18:17.263145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 74/141] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:17 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:17.427198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-790de59a-1080-47e0-9939-93d721352827 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:17.428512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-790de59a-1080-47e0-9939-93d721352827 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:17.442013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-790de59a-1080-47e0-9939-93d721352827 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:17.442771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-790de59a-1080-47e0-9939-93d721352827 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:17.443663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-790de59a-1080-47e0-9939-93d721352827 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:17.444163 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-790de59a-1080-47e0-9939-93d721352827 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:17.811124 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-790de59a-1080-47e0-9939-93d721352827 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:17.812715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-da598fd2-59e7-4783-8fe5-8c1105d4946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:17.813755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-da598fd2-59e7-4783-8fe5-8c1105d4946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:17.821813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da598fd2-59e7-4783-8fe5-8c1105d4946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:17.823060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-790de59a-1080-47e0-9939-93d721352827 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1553 microversion: 2.1 time: 0.399769 May 25 16:18:17.823674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 68/142] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:17 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1553 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:17.825280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da598fd2-59e7-4783-8fe5-8c1105d4946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:17.826582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da598fd2-59e7-4783-8fe5-8c1105d4946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:17.827021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da598fd2-59e7-4783-8fe5-8c1105d4946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:17.839572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:17.840238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:17.849388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:17.865044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:17.867263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:17.867883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:18.087295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-da598fd2-59e7-4783-8fe5-8c1105d4946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:18.098865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-da598fd2-59e7-4783-8fe5-8c1105d4946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1558 microversion: 2.1 time: 0.290423 May 25 16:18:18.100286 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 75/143] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:17 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1558 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:18.108717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:18.117130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:18.117335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-de57cbf1-b041-41a6-ac4c-89650ab91cdb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:18.117905 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-de57cbf1-b041-41a6-ac4c-89650ab91cdb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:18.120662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:18.123668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-34bc11a8-9c81-4979-9b4c-3dafdfa1edb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1599 microversion: 2.20 time: 0.297500 May 25 16:18:18.124459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 69/144] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:18:17 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1599 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:18:18.124865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de57cbf1-b041-41a6-ac4c-89650ab91cdb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:18.125352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de57cbf1-b041-41a6-ac4c-89650ab91cdb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:18.126128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de57cbf1-b041-41a6-ac4c-89650ab91cdb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:18.126569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de57cbf1-b041-41a6-ac4c-89650ab91cdb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:18.141856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9126d4e2-49ca-42c4-b910-a4ff64c5bd71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:18.142447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9126d4e2-49ca-42c4-b910-a4ff64c5bd71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:18.149583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9126d4e2-49ca-42c4-b910-a4ff64c5bd71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:18.150553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9126d4e2-49ca-42c4-b910-a4ff64c5bd71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:18.151486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9126d4e2-49ca-42c4-b910-a4ff64c5bd71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:18.152223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9126d4e2-49ca-42c4-b910-a4ff64c5bd71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:18.228192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-de57cbf1-b041-41a6-ac4c-89650ab91cdb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.114927 May 25 16:18:18.228867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 76/145] 10.222.0.45 () {58 vars in 1274 bytes} [Wed May 25 16:18:18 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-interface => generated 285 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:18:18.281244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9126d4e2-49ca-42c4-b910-a4ff64c5bd71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.143163 May 25 16:18:18.282140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 70/146] 10.222.0.45 () {60 vars in 1315 bytes} [Wed May 25 16:18:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-interface => generated 284 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:18:18.285802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-df5a9a52-37ea-450d-9b95-52cedd12e304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:18.287126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-df5a9a52-37ea-450d-9b95-52cedd12e304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:18.297670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1338216132", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "networks": [{"uuid": "b13ad99f-3558-45d2-be4a-ca0dfa266747"}]}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:18.311975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df5a9a52-37ea-450d-9b95-52cedd12e304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:18.314544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df5a9a52-37ea-450d-9b95-52cedd12e304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:18.315456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df5a9a52-37ea-450d-9b95-52cedd12e304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:18.317042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df5a9a52-37ea-450d-9b95-52cedd12e304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:18.397840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] validate_networks() for [('b13ad99f-3558-45d2-be4a-ca0dfa266747', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:18:18.807435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-df5a9a52-37ea-450d-9b95-52cedd12e304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:18.819001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-df5a9a52-37ea-450d-9b95-52cedd12e304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.542243 May 25 16:18:18.819894 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 77/147] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:18 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:18.840722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5ccb0e03-4bdb-4cbd-89b6-ba38a50c8c53 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:18.843686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5ccb0e03-4bdb-4cbd-89b6-ba38a50c8c53 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:18.857666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ccb0e03-4bdb-4cbd-89b6-ba38a50c8c53 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:18.858450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ccb0e03-4bdb-4cbd-89b6-ba38a50c8c53 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:18.859768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ccb0e03-4bdb-4cbd-89b6-ba38a50c8c53 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:18.861871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ccb0e03-4bdb-4cbd-89b6-ba38a50c8c53 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:20.047255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5ccb0e03-4bdb-4cbd-89b6-ba38a50c8c53 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:20.055858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5ccb0e03-4bdb-4cbd-89b6-ba38a50c8c53 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1553 microversion: 2.1 time: 1.219850 May 25 16:18:20.068133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 78/148] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:18 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1553 bytes in 1232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:20.078176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-9001edc3-0685-40da-90a2-950986674ad2 req-492a54ce-cbeb-434f-ad1d-44445c504610 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48341208-161c-454c-b88e-56a30f695cb7", "tag": "e2ff0c6b-df5c-4cfe-b8f2-13db43352e9b"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:20.093858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-9001edc3-0685-40da-90a2-950986674ad2 req-492a54ce-cbeb-434f-ad1d-44445c504610 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:20.094373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-9001edc3-0685-40da-90a2-950986674ad2 req-492a54ce-cbeb-434f-ad1d-44445c504610 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:20.109046 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-9001edc3-0685-40da-90a2-950986674ad2 req-492a54ce-cbeb-434f-ad1d-44445c504610 service nova] Creating event network-changed:e2ff0c6b-df5c-4cfe-b8f2-13db43352e9b for instance 48341208-161c-454c-b88e-56a30f695cb7 on hv-cinder-84327 May 25 16:18:20.116554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-9001edc3-0685-40da-90a2-950986674ad2 req-492a54ce-cbeb-434f-ad1d-44445c504610 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043737 May 25 16:18:20.117848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 79/149] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:20.126268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b9fd30c0-71f9-4dba-94ae-0f3073747368 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:20.127339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b9fd30c0-71f9-4dba-94ae-0f3073747368 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:20.135709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9fd30c0-71f9-4dba-94ae-0f3073747368 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:20.136419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9fd30c0-71f9-4dba-94ae-0f3073747368 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:20.137391 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9fd30c0-71f9-4dba-94ae-0f3073747368 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:20.137977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9fd30c0-71f9-4dba-94ae-0f3073747368 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:20.763967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:18:20.764553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:18:20.764871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:18:20.765342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:18:20.765781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:18:20.775520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:20.776209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:20.777003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:20.777435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:20.800661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Getting quotas for project 2aca09e1a3bc4926ba6c14f975728e39. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:18:20.804993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Getting quotas for user 63c237a15880463d9b3850dac6413ceb and project 2aca09e1a3bc4926ba6c14f975728e39. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:18:20.818340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:18:20.823829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:18:20.899412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-889c9881-beb5-4e46-991a-5218df77cf9b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.605052 May 25 16:18:20.907096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 71/150] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:18:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 2614 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:18:20.914605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-780cc5c0-d331-471f-bca8-d83a8ceb715c req-513049ee-380b-4878-8968-17d9ed1484f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "07b98495-1c75-4de3-a9ff-0303003df4e2", "name": "network-vif-unplugged", "status": "completed", "tag": "5883de34-5df8-4619-9f45-34c47caa0f75"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:20.925514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-780cc5c0-d331-471f-bca8-d83a8ceb715c req-513049ee-380b-4878-8968-17d9ed1484f4 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:20.926307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-780cc5c0-d331-471f-bca8-d83a8ceb715c req-513049ee-380b-4878-8968-17d9ed1484f4 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:20.940099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-780cc5c0-d331-471f-bca8-d83a8ceb715c req-513049ee-380b-4878-8968-17d9ed1484f4 service nova] Creating event network-vif-unplugged:5883de34-5df8-4619-9f45-34c47caa0f75 for instance 07b98495-1c75-4de3-a9ff-0303003df4e2 on hv-cinder-84327 May 25 16:18:20.946952 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-780cc5c0-d331-471f-bca8-d83a8ceb715c req-513049ee-380b-4878-8968-17d9ed1484f4 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036496 May 25 16:18:20.947847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 72/151] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:20 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:20.953372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f8df89cc-858a-4ffe-bf73-62fe1996f9e0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:20.954312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f8df89cc-858a-4ffe-bf73-62fe1996f9e0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:21.058803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b9fd30c0-71f9-4dba-94ae-0f3073747368 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:21.075880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b9fd30c0-71f9-4dba-94ae-0f3073747368 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.955445 May 25 16:18:21.089399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 80/152] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:20 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 970 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:21.096084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9dfca4ef-d823-44c3-8ed2-917187345f5d tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:21.096869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9dfca4ef-d823-44c3-8ed2-917187345f5d tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:21.104699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dfca4ef-d823-44c3-8ed2-917187345f5d tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:21.105589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dfca4ef-d823-44c3-8ed2-917187345f5d tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:21.108607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dfca4ef-d823-44c3-8ed2-917187345f5d tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:21.109135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dfca4ef-d823-44c3-8ed2-917187345f5d tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:21.119668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f8df89cc-858a-4ffe-bf73-62fe1996f9e0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:21.129608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f8df89cc-858a-4ffe-bf73-62fe1996f9e0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1214 microversion: 2.1 time: 0.179190 May 25 16:18:21.131564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 73/153] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:20 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1214 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:21.970554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9dfca4ef-d823-44c3-8ed2-917187345f5d tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:21.979335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9dfca4ef-d823-44c3-8ed2-917187345f5d tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1553 microversion: 2.1 time: 0.888852 May 25 16:18:21.980128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 81/154] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:21 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1553 bytes in 890 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:22.092304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2f9560e4-2038-4a3d-aed0-56862438e57e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:22.092890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2f9560e4-2038-4a3d-aed0-56862438e57e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:22.100220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f9560e4-2038-4a3d-aed0-56862438e57e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:22.100761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f9560e4-2038-4a3d-aed0-56862438e57e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:22.101752 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f9560e4-2038-4a3d-aed0-56862438e57e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:22.102253 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f9560e4-2038-4a3d-aed0-56862438e57e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:22.148383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a5afd845-e6a4-48d2-bcdd-35dca4b6af82 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:22.148964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a5afd845-e6a4-48d2-bcdd-35dca4b6af82 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:22.155754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5afd845-e6a4-48d2-bcdd-35dca4b6af82 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:22.156300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5afd845-e6a4-48d2-bcdd-35dca4b6af82 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:22.157075 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5afd845-e6a4-48d2-bcdd-35dca4b6af82 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:22.157510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5afd845-e6a4-48d2-bcdd-35dca4b6af82 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:22.340976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a5afd845-e6a4-48d2-bcdd-35dca4b6af82 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:22.353052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a5afd845-e6a4-48d2-bcdd-35dca4b6af82 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1274 microversion: 2.1 time: 0.207920 May 25 16:18:22.354702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 82/155] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:22 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1274 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:22.362478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-bc4f9776-10aa-47d0-8e95-b84551899499 req-3fc3d2ad-49ed-4c5f-87b7-2c0697863659 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "07b98495-1c75-4de3-a9ff-0303003df4e2", "name": "network-vif-deleted", "tag": "5883de34-5df8-4619-9f45-34c47caa0f75"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:22.370459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-bc4f9776-10aa-47d0-8e95-b84551899499 req-3fc3d2ad-49ed-4c5f-87b7-2c0697863659 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:22.370935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-bc4f9776-10aa-47d0-8e95-b84551899499 req-3fc3d2ad-49ed-4c5f-87b7-2c0697863659 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:22.385586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-bc4f9776-10aa-47d0-8e95-b84551899499 req-3fc3d2ad-49ed-4c5f-87b7-2c0697863659 service nova] Creating event network-vif-deleted:5883de34-5df8-4619-9f45-34c47caa0f75 for instance 07b98495-1c75-4de3-a9ff-0303003df4e2 on hv-cinder-84327 May 25 16:18:22.390632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-bc4f9776-10aa-47d0-8e95-b84551899499 req-3fc3d2ad-49ed-4c5f-87b7-2c0697863659 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033043 May 25 16:18:22.391316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 83/156] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:22.622813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2f9560e4-2038-4a3d-aed0-56862438e57e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:22.636537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2f9560e4-2038-4a3d-aed0-56862438e57e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.547999 May 25 16:18:22.637784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 74/157] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:22 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:23.000643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:23.001348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:23.022054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:23.022592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:23.023418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:23.023927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:23.111487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:23.112142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:23.115846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lazy-loading 'fault' on Instance uuid 07b98495-1c75-4de3-a9ff-0303003df4e2 {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:18:23.373257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-69b9b0df-bb2b-4262-a636-50547d9cbcaf tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:23.373902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-69b9b0df-bb2b-4262-a636-50547d9cbcaf tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:23.383236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69b9b0df-bb2b-4262-a636-50547d9cbcaf tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:23.383601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69b9b0df-bb2b-4262-a636-50547d9cbcaf tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:23.384368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69b9b0df-bb2b-4262-a636-50547d9cbcaf tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:23.384796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69b9b0df-bb2b-4262-a636-50547d9cbcaf tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:23.583622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-69b9b0df-bb2b-4262-a636-50547d9cbcaf tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:23.583790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1cb5512dbadb4b70b030e7e3bb297dad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabcb11a7be746719029396d4abab6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:23.592289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-69b9b0df-bb2b-4262-a636-50547d9cbcaf tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1272 microversion: 2.1 time: 0.223526 May 25 16:18:23.592429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-72daf999-943e-45ae-b286-9cb99c2bb5ea tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 200 len: 1305 microversion: 2.1 time: 0.596594 May 25 16:18:23.593473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 84/158] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:22 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 1305 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:23.594864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 75/159] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:23 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1272 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:23.659084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cca2ca37-dc6c-4f52-bbc1-c83f11a686ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:23.660127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cca2ca37-dc6c-4f52-bbc1-c83f11a686ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:23.667136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cca2ca37-dc6c-4f52-bbc1-c83f11a686ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:23.667630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cca2ca37-dc6c-4f52-bbc1-c83f11a686ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:23.668539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cca2ca37-dc6c-4f52-bbc1-c83f11a686ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:23.668975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cca2ca37-dc6c-4f52-bbc1-c83f11a686ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:23.950475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cca2ca37-dc6c-4f52-bbc1-c83f11a686ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:23.958797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cca2ca37-dc6c-4f52-bbc1-c83f11a686ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.308228 May 25 16:18:23.959665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 85/160] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:23 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:24.424445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-c120c2a7-186e-45f1-9cbe-c7c8cbbf069a req-ad702c5d-8bfd-47b0-b0a8-1fad1356575a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4", "tag": "4d5ed500-2623-48b8-bc81-a72eb4ffeb2d"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:24.432929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-c120c2a7-186e-45f1-9cbe-c7c8cbbf069a req-ad702c5d-8bfd-47b0-b0a8-1fad1356575a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:24.433348 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-c120c2a7-186e-45f1-9cbe-c7c8cbbf069a req-ad702c5d-8bfd-47b0-b0a8-1fad1356575a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:24.458433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-c120c2a7-186e-45f1-9cbe-c7c8cbbf069a req-ad702c5d-8bfd-47b0-b0a8-1fad1356575a service nova] Creating event network-changed:4d5ed500-2623-48b8-bc81-a72eb4ffeb2d for instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 on hv-cinder-84327 May 25 16:18:24.464470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-c120c2a7-186e-45f1-9cbe-c7c8cbbf069a req-ad702c5d-8bfd-47b0-b0a8-1fad1356575a service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043826 May 25 16:18:24.465087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 76/161] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:24.614148 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:24.615367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3c222ec0-8771-4901-bedc-46572a5bef73 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:24.616052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:24.616163 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3c222ec0-8771-4901-bedc-46572a5bef73 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:24.627284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3c222ec0-8771-4901-bedc-46572a5bef73 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:24.627757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3c222ec0-8771-4901-bedc-46572a5bef73 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:24.628530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3c222ec0-8771-4901-bedc-46572a5bef73 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:24.628968 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3c222ec0-8771-4901-bedc-46572a5bef73 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:24.630976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:24.635747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:24.637242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:24.646019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:24.684070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:24.689249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] HTTP exception thrown: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Request method failure captured: May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 HTTP/1.1 May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaEpZzTG4VxUENalDzv3iSTRInkSM3qNPkC179Ros3vEYAeCNgEI2-n71Ox8XkC7XYZFOytyLaojN2qIedOUm6jOFtZt-cXuSNVZ0r8Cys5mqqBUh4IZToawWJIys8cwx3F-p-C32FvE8KohyPXqJ0DlYd7psFjbcH2zpvOGAtnOAU May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: fabcb11a7be746719029396d4abab6cc May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-DeleteServersTestJSON-264384839 May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:24.705986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_fabcb11a7be746719029396d4abab6cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-DeleteServersTestJSON-264384839 May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: fabcb11a7be746719029396d4abab6cc May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-DeleteServersTestJSON-264384839 May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-DeleteServersTestJSON-264384839-project May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1cb5512dbadb4b70b030e7e3bb297dad May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-DeleteServersTestJSON-264384839-project May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.707325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '07b98495-1c75-4de3-a9ff-0303003df4e2'} May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:24.708022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.709136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.713652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Returning 404 to user: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:24.717100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d2781140-3f2d-43af-8c2a-84d6636ab1f3 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 404 len: 111 microversion: 2.1 time: 0.108908 May 25 16:18:24.738145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 86/162] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:24 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 111 bytes in 131 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:18:24.880312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:24.882572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:24.892314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:24.892979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:24.894033 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:24.894679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:24.912520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] HTTP exception thrown: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Request method failure captured: May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2/os-volume_attachments HTTP/1.1 May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaEpZzTG4VxUENalDzv3iSTRInkSM3qNPkC179Ros3vEYAeCNgEI2-n71Ox8XkC7XYZFOytyLaojN2qIedOUm6jOFtZt-cXuSNVZ0r8Cys5mqqBUh4IZToawWJIys8cwx3F-p-C32FvE8KohyPXqJ0DlYd7psFjbcH2zpvOGAtnOAU May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: fabcb11a7be746719029396d4abab6cc May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-DeleteServersTestJSON-264384839 May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:24.914860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_fabcb11a7be746719029396d4abab6cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-DeleteServersTestJSON-264384839 May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: fabcb11a7be746719029396d4abab6cc May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-DeleteServersTestJSON-264384839 May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-DeleteServersTestJSON-264384839-project May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1cb5512dbadb4b70b030e7e3bb297dad May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-DeleteServersTestJSON-264384839-project May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.916130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': '07b98495-1c75-4de3-a9ff-0303003df4e2'} May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:24.917202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:24.918238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:24.919366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Returning 404 to user: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:24.919366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f43a9f7e-17f6-4b59-9669-5dc864b04895 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.041038 May 25 16:18:24.921221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 87/163] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:18:24 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2/os-volume_attachments => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 25 16:18:24.977699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-94b07f61-b8d4-4d46-8089-92b11a6f8c79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:24.978313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-94b07f61-b8d4-4d46-8089-92b11a6f8c79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:24.990685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94b07f61-b8d4-4d46-8089-92b11a6f8c79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:24.992231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94b07f61-b8d4-4d46-8089-92b11a6f8c79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:24.995201 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94b07f61-b8d4-4d46-8089-92b11a6f8c79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:24.997766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94b07f61-b8d4-4d46-8089-92b11a6f8c79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:25.105554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3c222ec0-8771-4901-bedc-46572a5bef73 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:25.117120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3c222ec0-8771-4901-bedc-46572a5bef73 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1314 microversion: 2.1 time: 0.506209 May 25 16:18:25.119591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 77/164] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:24 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1314 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:25.374313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-94b07f61-b8d4-4d46-8089-92b11a6f8c79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:25.391388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-94b07f61-b8d4-4d46-8089-92b11a6f8c79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.409783 May 25 16:18:25.392044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 88/165] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:24 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:26.136444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-828a65d8-bde3-4491-a1ee-4470e514b54c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:26.137046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-828a65d8-bde3-4491-a1ee-4470e514b54c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:26.143611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-828a65d8-bde3-4491-a1ee-4470e514b54c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:26.144358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-828a65d8-bde3-4491-a1ee-4470e514b54c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:26.145335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-828a65d8-bde3-4491-a1ee-4470e514b54c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:26.145978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-828a65d8-bde3-4491-a1ee-4470e514b54c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:26.271558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:26.272084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:26.278642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:26.279431 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:26.280353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:26.281236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:26.300093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] HTTP exception thrown: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Request method failure captured: May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 HTTP/1.1 May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaEpZzTG4VxUENalDzv3iSTRInkSM3qNPkC179Ros3vEYAeCNgEI2-n71Ox8XkC7XYZFOytyLaojN2qIedOUm6jOFtZt-cXuSNVZ0r8Cys5mqqBUh4IZToawWJIys8cwx3F-p-C32FvE8KohyPXqJ0DlYd7psFjbcH2zpvOGAtnOAU May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: fabcb11a7be746719029396d4abab6cc May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-DeleteServersTestJSON-264384839 May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:26.301903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_fabcb11a7be746719029396d4abab6cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-DeleteServersTestJSON-264384839 May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: fabcb11a7be746719029396d4abab6cc May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-DeleteServersTestJSON-264384839 May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-DeleteServersTestJSON-264384839-project May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1cb5512dbadb4b70b030e7e3bb297dad May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-DeleteServersTestJSON-264384839-project May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.303049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '07b98495-1c75-4de3-a9ff-0303003df4e2'} May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:26.303724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.304743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.307261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Returning 404 to user: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:26.307261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-54f2ea61-311a-4b56-9169-a73454e33256 tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "DELETE /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 404 len: 111 microversion: 2.1 time: 0.037021 May 25 16:18:26.307261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 89/166] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:18:26 2022] DELETE /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:18:26.314534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:26.315344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] [instance: 07b98495-1c75-4de3-a9ff-0303003df4e2] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:26.318173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-828a65d8-bde3-4491-a1ee-4470e514b54c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:26.322402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:26.323132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:26.324186 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:26.325409 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:26.336427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-828a65d8-bde3-4491-a1ee-4470e514b54c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1314 microversion: 2.1 time: 0.203844 May 25 16:18:26.337660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 78/167] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:26 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1314 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:26.344024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] HTTP exception thrown: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Request method failure captured: May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 HTTP/1.1 May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaEpZzTG4VxUENalDzv3iSTRInkSM3qNPkC179Ros3vEYAeCNgEI2-n71Ox8XkC7XYZFOytyLaojN2qIedOUm6jOFtZt-cXuSNVZ0r8Cys5mqqBUh4IZToawWJIys8cwx3F-p-C32FvE8KohyPXqJ0DlYd7psFjbcH2zpvOGAtnOAU May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: fabcb11a7be746719029396d4abab6cc May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-DeleteServersTestJSON-264384839 May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:26.345558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_fabcb11a7be746719029396d4abab6cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/fabcb11a7be746719029396d4abab6cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-DeleteServersTestJSON-264384839 May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: fabcb11a7be746719029396d4abab6cc May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-DeleteServersTestJSON-264384839 May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-DeleteServersTestJSON-264384839-project May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1cb5512dbadb4b70b030e7e3bb297dad May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-DeleteServersTestJSON-264384839-project May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.346991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '07b98495-1c75-4de3-a9ff-0303003df4e2'} May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:26.347730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. May 25 16:18:26.348917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:26.349838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] Returning 404 to user: Instance 07b98495-1c75-4de3-a9ff-0303003df4e2 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:26.349838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a0c8bee7-7c0c-43f2-879f-312a3de872df tempest-DeleteServersTestJSON-264384839 tempest-DeleteServersTestJSON-264384839-project] 10.222.0.45 "GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2" status: 404 len: 111 microversion: 2.1 time: 0.036573 May 25 16:18:26.350777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 90/168] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:26 2022] GET /compute/v2.1/servers/07b98495-1c75-4de3-a9ff-0303003df4e2 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:18:26.409906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-84302af2-3241-4064-b5b0-123f5f489e00 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:26.410796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-84302af2-3241-4064-b5b0-123f5f489e00 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:26.421632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84302af2-3241-4064-b5b0-123f5f489e00 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:26.422070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84302af2-3241-4064-b5b0-123f5f489e00 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:26.427430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84302af2-3241-4064-b5b0-123f5f489e00 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:26.427898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84302af2-3241-4064-b5b0-123f5f489e00 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:26.503754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-76bfb950-2bcd-4fed-b902-dc59e7a9b37b req-ffcdb6e8-0717-4e4b-bbd2-f3508ec988bd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f674eae5-f61a-4079-b358-66339ea78ca8", "tag": "bc347387-d953-4b27-aa60-a307fe381a78"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:26.520309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-76bfb950-2bcd-4fed-b902-dc59e7a9b37b req-ffcdb6e8-0717-4e4b-bbd2-f3508ec988bd service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:26.521061 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-76bfb950-2bcd-4fed-b902-dc59e7a9b37b req-ffcdb6e8-0717-4e4b-bbd2-f3508ec988bd service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:26.552650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-76bfb950-2bcd-4fed-b902-dc59e7a9b37b req-ffcdb6e8-0717-4e4b-bbd2-f3508ec988bd service nova] Creating event network-changed:bc347387-d953-4b27-aa60-a307fe381a78 for instance f674eae5-f61a-4079-b358-66339ea78ca8 on hv-cinder-84327 May 25 16:18:26.559374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-76bfb950-2bcd-4fed-b902-dc59e7a9b37b req-ffcdb6e8-0717-4e4b-bbd2-f3508ec988bd service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059455 May 25 16:18:26.560648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 91/169] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:26.732617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-84302af2-3241-4064-b5b0-123f5f489e00 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:26.751275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-84302af2-3241-4064-b5b0-123f5f489e00 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.342069 May 25 16:18:26.753867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 79/170] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:26 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:27.354463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ce07708c-54ac-4fcd-9bcf-58a842879736 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:27.355866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ce07708c-54ac-4fcd-9bcf-58a842879736 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:27.362495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ce07708c-54ac-4fcd-9bcf-58a842879736 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:27.363836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ce07708c-54ac-4fcd-9bcf-58a842879736 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:27.364797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ce07708c-54ac-4fcd-9bcf-58a842879736 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:27.365434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ce07708c-54ac-4fcd-9bcf-58a842879736 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:27.770276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-62eaec00-5d6c-4513-a14c-21755f470dff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:27.771532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-62eaec00-5d6c-4513-a14c-21755f470dff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:27.783313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62eaec00-5d6c-4513-a14c-21755f470dff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:27.784070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62eaec00-5d6c-4513-a14c-21755f470dff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:27.785004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62eaec00-5d6c-4513-a14c-21755f470dff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:27.786274 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62eaec00-5d6c-4513-a14c-21755f470dff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:27.872809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ce07708c-54ac-4fcd-9bcf-58a842879736 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:27.885635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ce07708c-54ac-4fcd-9bcf-58a842879736 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1314 microversion: 2.1 time: 0.535425 May 25 16:18:27.891552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 92/171] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:27 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1314 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:28.267717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-62eaec00-5d6c-4513-a14c-21755f470dff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:28.278948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-62eaec00-5d6c-4513-a14c-21755f470dff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.512647 May 25 16:18:28.279573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 80/172] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:27 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:28.902455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-27d09c72-5247-4736-ac70-87f575e19a61 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:28.903063 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-27d09c72-5247-4736-ac70-87f575e19a61 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:28.910078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-27d09c72-5247-4736-ac70-87f575e19a61 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:28.910766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-27d09c72-5247-4736-ac70-87f575e19a61 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:28.912216 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-27d09c72-5247-4736-ac70-87f575e19a61 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:28.912846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-27d09c72-5247-4736-ac70-87f575e19a61 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:29.301489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4437b3ab-37f7-40d4-9359-392e1c77dce4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:29.302148 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4437b3ab-37f7-40d4-9359-392e1c77dce4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:29.319699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4437b3ab-37f7-40d4-9359-392e1c77dce4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:29.321158 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4437b3ab-37f7-40d4-9359-392e1c77dce4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:29.323500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4437b3ab-37f7-40d4-9359-392e1c77dce4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:29.324311 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4437b3ab-37f7-40d4-9359-392e1c77dce4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:29.411396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:29.430488 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-27d09c72-5247-4736-ac70-87f575e19a61 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:29.447801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-27d09c72-5247-4736-ac70-87f575e19a61 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1314 microversion: 2.1 time: 0.549230 May 25 16:18:29.453137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 93/173] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:28 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1314 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:29.648604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8792ce49-bb6a-4a43-82be-089052fad989 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Action: 'create', calling method: .version_select at 0x7f4a301ad310>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-337875140", "metadata": {"Type": "work"}}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8792ce49-bb6a-4a43-82be-089052fad989 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Request method failure captured: May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 121 May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbVeJcB3vSIQPIjXii6KNH6aTM3N_3e5sF_gUSlj1j9LKb0edKxzghYhUQvhkWpcXATEWLfQFUkQ8FQXQ_7lgWxOCSlXWP7CCzbLJajIOkcios3WvHz5aJNxFb-05jTRHoP5dQSQyT5qbEykZkmzy3jJVKYnXE7StTZywvuEbW4umo May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:29.655891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1b3c35d89f2b498ea993ecc4cf4f5268 May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-337875140", "metadata": {"Type": "work"}}} May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: May 25 16:18:29.657144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-337875140', 'metadata': {'Type': 'work'}}}} May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise error May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi On instance['volume']: May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi '#$%' May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:29.657865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 25 16:18:29.659088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 25 16:18:29.660705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 25 16:18:29.660705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.662989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8792ce49-bb6a-4a43-82be-089052fad989 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:29.665039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8792ce49-bb6a-4a43-82be-089052fad989 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] 10.222.0.45 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.197523 May 25 16:18:29.666892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 94/174] 10.222.0.45 () {60 vars in 1155 bytes} [Wed May 25 16:18:29 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 200 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:18:29.677332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b7b8691f-abde-4405-9436-bc1ef4af9963 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Action: 'create', calling method: .version_select at 0x7f4a3031c550>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1283581595", "metadata": {"Type": "work"}}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b7b8691f-abde-4405-9436-bc1ef4af9963 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Request method failure captured: May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 120 May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbVeJcB3vSIQPIjXii6KNH6aTM3N_3e5sF_gUSlj1j9LKb0edKxzghYhUQvhkWpcXATEWLfQFUkQ8FQXQ_7lgWxOCSlXWP7CCzbLJajIOkcios3WvHz5aJNxFb-05jTRHoP5dQSQyT5qbEykZkmzy3jJVKYnXE7StTZywvuEbW4umo May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:29.680686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1b3c35d89f2b498ea993ecc4cf4f5268 May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1283581595", "metadata": {"Type": "work"}}} May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: May 25 16:18:29.681937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1283581595', 'metadata': {'Type': 'work'}}}} May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise error May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi On instance['volume']: May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi '0' May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:29.682628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 25 16:18:29.683881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 25 16:18:29.684910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 25 16:18:29.684910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.684910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b7b8691f-abde-4405-9436-bc1ef4af9963 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:29.684910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b7b8691f-abde-4405-9436-bc1ef4af9963 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] 10.222.0.45 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.010269 May 25 16:18:29.684910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 95/175] 10.222.0.45 () {60 vars in 1155 bytes} [Wed May 25 16:18:29 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:18:29.694202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4fc59f71-960b-40f0-8a50-4b79f1d161b6 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Action: 'create', calling method: .version_select at 0x7f4a30453b80>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2135647697", "metadata": {"Type": "work"}}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4fc59f71-960b-40f0-8a50-4b79f1d161b6 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Request method failure captured: May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 119 May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbVeJcB3vSIQPIjXii6KNH6aTM3N_3e5sF_gUSlj1j9LKb0edKxzghYhUQvhkWpcXATEWLfQFUkQ8FQXQ_7lgWxOCSlXWP7CCzbLJajIOkcios3WvHz5aJNxFb-05jTRHoP5dQSQyT5qbEykZkmzy3jJVKYnXE7StTZywvuEbW4umo May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:29.696661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1b3c35d89f2b498ea993ecc4cf4f5268 May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2135647697", "metadata": {"Type": "work"}}} May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: May 25 16:18:29.697785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-2135647697', 'metadata': {'Type': 'work'}}}} May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise error May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi On instance['volume']: May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi '' May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:29.698463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 25 16:18:29.699768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 25 16:18:29.700746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 25 16:18:29.700746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.700746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4fc59f71-960b-40f0-8a50-4b79f1d161b6 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:29.700746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4fc59f71-960b-40f0-8a50-4b79f1d161b6 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] 10.222.0.45 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.008981 May 25 16:18:29.702742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 96/176] 10.222.0.45 () {60 vars in 1155 bytes} [Wed May 25 16:18:29 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:18:29.708513 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e7af4f70-9531-4b07-9d5b-6c1087da123c tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Calling method '.version_select at 0x7f4a301ad700>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:29.742073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-e7af4f70-9531-4b07-9d5b-6c1087da123c tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] HTTP exception thrown: Volume tempest-invalid-1953776615 could not be found. May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e7af4f70-9531-4b07-9d5b-6c1087da123c tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Request method failure captured: May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1953776615 HTTP/1.1 May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbVeJcB3vSIQPIjXii6KNH6aTM3N_3e5sF_gUSlj1j9LKb0edKxzghYhUQvhkWpcXATEWLfQFUkQ8FQXQ_7lgWxOCSlXWP7CCzbLJajIOkcios3WvHz5aJNxFb-05jTRHoP5dQSQyT5qbEykZkmzy3jJVKYnXE7StTZywvuEbW4umo May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:29.746226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1b3c35d89f2b498ea993ecc4cf4f5268 May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Volume tempest-invalid-1953776615 could not be found. May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'tempest-invalid-1953776615'} May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:29.747436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1953776615 could not be found. (HTTP 404) (Request-ID: req-18da4cc8-be31-42aa-a532-cadfc26df724) May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.748180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 25 16:18:29.749163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1953776615 could not be found. May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:18:29.750082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1953776615 could not be found. May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1953776615 could not be found. May 25 16:18:29.751106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.753910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e7af4f70-9531-4b07-9d5b-6c1087da123c tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Returning 404 to user: Volume tempest-invalid-1953776615 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:29.755036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e7af4f70-9531-4b07-9d5b-6c1087da123c tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] 10.222.0.45 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1953776615" status: 404 len: 99 microversion: 2.1 time: 0.050999 May 25 16:18:29.755855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 97/177] 10.222.0.45 () {58 vars in 1217 bytes} [Wed May 25 16:18:29 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1953776615 => generated 99 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) May 25 16:18:29.768431 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3bf9f702-5807-499d-98d7-5564db46fdd2 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] 10.222.0.45 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005032 May 25 16:18:29.769055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 98/178] 10.222.0.45 () {58 vars in 1139 bytes} [Wed May 25 16:18:29 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) May 25 16:18:29.778147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-43926677-1892-4bd0-89ba-f78270bddcab tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003777 May 25 16:18:29.778728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 99/179] 10.222.0.45 () {58 vars in 1136 bytes} [Wed May 25 16:18:29 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) May 25 16:18:29.787967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-316475b5-0ace-4d7b-9725-e08e8cc2e085 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Calling method '.version_select at 0x7f4a302b20d0>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:29.830154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-316475b5-0ace-4d7b-9725-e08e8cc2e085 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] HTTP exception thrown: Volume 53a10428-b35c-444f-9290-40805f562ab4 could not be found. May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-316475b5-0ace-4d7b-9725-e08e8cc2e085 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Request method failure captured: May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/os-volumes/53a10428-b35c-444f-9290-40805f562ab4 HTTP/1.1 May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbVeJcB3vSIQPIjXii6KNH6aTM3N_3e5sF_gUSlj1j9LKb0edKxzghYhUQvhkWpcXATEWLfQFUkQ8FQXQ_7lgWxOCSlXWP7CCzbLJajIOkcios3WvHz5aJNxFb-05jTRHoP5dQSQyT5qbEykZkmzy3jJVKYnXE7StTZywvuEbW4umo May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:29.832692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1b3c35d89f2b498ea993ecc4cf4f5268 May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Volume 53a10428-b35c-444f-9290-40805f562ab4 could not be found. May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '53a10428-b35c-444f-9290-40805f562ab4'} May 25 16:18:29.834027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 53a10428-b35c-444f-9290-40805f562ab4 could not be found. (HTTP 404) (Request-ID: req-a143341d-2e69-48b0-b20d-b0f11103c967) May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.835421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 25 16:18:29.836591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 53a10428-b35c-444f-9290-40805f562ab4 could not be found. May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 25 16:18:29.837764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 53a10428-b35c-444f-9290-40805f562ab4 could not be found. May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 53a10428-b35c-444f-9290-40805f562ab4 could not be found. May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-316475b5-0ace-4d7b-9725-e08e8cc2e085 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Returning 404 to user: Volume 53a10428-b35c-444f-9290-40805f562ab4 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-316475b5-0ace-4d7b-9725-e08e8cc2e085 tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] 10.222.0.45 "DELETE /compute/v2.1/os-volumes/53a10428-b35c-444f-9290-40805f562ab4" status: 404 len: 109 microversion: 2.1 time: 0.052780 May 25 16:18:29.838944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 100/180] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:18:29 2022] DELETE /compute/v2.1/os-volumes/53a10428-b35c-444f-9290-40805f562ab4 => generated 109 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:18:29.848472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c8fd5ca5-1224-49b2-a281-0036b1ec682f tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Calling method '.version_select at 0x7f4a302b2790>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:29.882956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-c8fd5ca5-1224-49b2-a281-0036b1ec682f tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] HTTP exception thrown: Volume 7c8fc75e-353e-4a53-b794-b89bcef50866 could not be found. May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c8fd5ca5-1224-49b2-a281-0036b1ec682f tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Request method failure captured: May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/os-volumes/7c8fc75e-353e-4a53-b794-b89bcef50866 HTTP/1.1 May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbVeJcB3vSIQPIjXii6KNH6aTM3N_3e5sF_gUSlj1j9LKb0edKxzghYhUQvhkWpcXATEWLfQFUkQ8FQXQ_7lgWxOCSlXWP7CCzbLJajIOkcios3WvHz5aJNxFb-05jTRHoP5dQSQyT5qbEykZkmzy3jJVKYnXE7StTZywvuEbW4umo May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:29.884974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/7ca5b81a34724b8c977ce0497e363b4f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 7ca5b81a34724b8c977ce0497e363b4f May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesNegativeTest-42817269 May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 1b3c35d89f2b498ea993ecc4cf4f5268 May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesNegativeTest-42817269-project May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Volume 7c8fc75e-353e-4a53-b794-b89bcef50866 could not be found. May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '7c8fc75e-353e-4a53-b794-b89bcef50866'} May 25 16:18:29.886481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi item = cinderclient( May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7c8fc75e-353e-4a53-b794-b89bcef50866 could not be found. (HTTP 404) (Request-ID: req-9b8ea72b-923c-4f56-8197-55f2ca7c9ea9) May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.887245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi item = cinderclient( May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:18:29.888252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7c8fc75e-353e-4a53-b794-b89bcef50866 could not be found. May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 25 16:18:29.889228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7c8fc75e-353e-4a53-b794-b89bcef50866 could not be found. May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7c8fc75e-353e-4a53-b794-b89bcef50866 could not be found. May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c8fd5ca5-1224-49b2-a281-0036b1ec682f tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] Returning 404 to user: Volume 7c8fc75e-353e-4a53-b794-b89bcef50866 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c8fd5ca5-1224-49b2-a281-0036b1ec682f tempest-VolumesNegativeTest-42817269 tempest-VolumesNegativeTest-42817269-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/7c8fc75e-353e-4a53-b794-b89bcef50866" status: 404 len: 109 microversion: 2.1 time: 0.044043 May 25 16:18:29.892518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 101/181] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:18:29 2022] GET /compute/v2.1/os-volumes/7c8fc75e-353e-4a53-b794-b89bcef50866 => generated 109 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:18:30.170430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4437b3ab-37f7-40d4-9359-392e1c77dce4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:30.179835 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4437b3ab-37f7-40d4-9359-392e1c77dce4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.885641 May 25 16:18:30.189653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 81/182] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:29 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 892 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:30.471298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b838217f-2389-4d3b-8f13-4e2d9d4b7ca0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:30.472563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b838217f-2389-4d3b-8f13-4e2d9d4b7ca0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:30.482269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b838217f-2389-4d3b-8f13-4e2d9d4b7ca0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:30.482972 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b838217f-2389-4d3b-8f13-4e2d9d4b7ca0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:30.484341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b838217f-2389-4d3b-8f13-4e2d9d4b7ca0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:30.484964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b838217f-2389-4d3b-8f13-4e2d9d4b7ca0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:30.696657 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b838217f-2389-4d3b-8f13-4e2d9d4b7ca0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:30.704688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b838217f-2389-4d3b-8f13-4e2d9d4b7ca0 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1314 microversion: 2.1 time: 0.238302 May 25 16:18:30.714781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 102/183] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:30 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1314 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:30.838140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-cc1b6e3c-fe03-45a5-8ff7-ec80e44943f1 req-f379b0e2-4a3c-44a6-805d-c1ade68b23ae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f674eae5-f61a-4079-b358-66339ea78ca8", "name": "network-vif-plugged", "status": "completed", "tag": "bc347387-d953-4b27-aa60-a307fe381a78"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:30.847757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:30.858322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-cc1b6e3c-fe03-45a5-8ff7-ec80e44943f1 req-f379b0e2-4a3c-44a6-805d-c1ade68b23ae service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:30.858829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-cc1b6e3c-fe03-45a5-8ff7-ec80e44943f1 req-f379b0e2-4a3c-44a6-805d-c1ade68b23ae service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:30.882197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-cc1b6e3c-fe03-45a5-8ff7-ec80e44943f1 req-f379b0e2-4a3c-44a6-805d-c1ade68b23ae service nova] Creating event network-vif-plugged:bc347387-d953-4b27-aa60-a307fe381a78 for instance f674eae5-f61a-4079-b358-66339ea78ca8 on hv-cinder-84327 May 25 16:18:30.891628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-cc1b6e3c-fe03-45a5-8ff7-ec80e44943f1 req-f379b0e2-4a3c-44a6-805d-c1ade68b23ae service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059244 May 25 16:18:30.892466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 82/184] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:31.198097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2d8f277a-4dc0-42c8-bd0f-a37dd420977b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:31.198794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2d8f277a-4dc0-42c8-bd0f-a37dd420977b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:31.206021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d8f277a-4dc0-42c8-bd0f-a37dd420977b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:31.206703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d8f277a-4dc0-42c8-bd0f-a37dd420977b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:31.207697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d8f277a-4dc0-42c8-bd0f-a37dd420977b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:31.208324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d8f277a-4dc0-42c8-bd0f-a37dd420977b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:31.494008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2d8f277a-4dc0-42c8-bd0f-a37dd420977b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:31.506777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2d8f277a-4dc0-42c8-bd0f-a37dd420977b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.309835 May 25 16:18:31.507651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 103/185] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:31 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:31.722954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3d2964de-0683-48a7-8a23-12bd6de60c56 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:31.723676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3d2964de-0683-48a7-8a23-12bd6de60c56 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:31.726025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=8 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:31.745143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3d2964de-0683-48a7-8a23-12bd6de60c56 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:31.745557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3d2964de-0683-48a7-8a23-12bd6de60c56 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:31.746322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3d2964de-0683-48a7-8a23-12bd6de60c56 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:31.746744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3d2964de-0683-48a7-8a23-12bd6de60c56 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:31.976490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3d2964de-0683-48a7-8a23-12bd6de60c56 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:31.986281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3d2964de-0683-48a7-8a23-12bd6de60c56 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1314 microversion: 2.1 time: 0.267882 May 25 16:18:31.987312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 83/186] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:31 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1314 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:32.529350 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9b571ac1-75e6-4f17-aefb-8ccdcc93baa2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:32.532348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9b571ac1-75e6-4f17-aefb-8ccdcc93baa2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:32.546890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b571ac1-75e6-4f17-aefb-8ccdcc93baa2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:32.551846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b571ac1-75e6-4f17-aefb-8ccdcc93baa2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:32.553032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b571ac1-75e6-4f17-aefb-8ccdcc93baa2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:32.553758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b571ac1-75e6-4f17-aefb-8ccdcc93baa2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:32.866283 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9b571ac1-75e6-4f17-aefb-8ccdcc93baa2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:32.880413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9b571ac1-75e6-4f17-aefb-8ccdcc93baa2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.360759 May 25 16:18:32.881007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 104/187] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:32 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:33.004443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7ddd1b32-f372-4592-8d05-f9b3de6dcc6b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:33.006025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7ddd1b32-f372-4592-8d05-f9b3de6dcc6b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:33.010454 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:33.023071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7ddd1b32-f372-4592-8d05-f9b3de6dcc6b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:33.026967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7ddd1b32-f372-4592-8d05-f9b3de6dcc6b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:33.028174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7ddd1b32-f372-4592-8d05-f9b3de6dcc6b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:33.028977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7ddd1b32-f372-4592-8d05-f9b3de6dcc6b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:33.336932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7ddd1b32-f372-4592-8d05-f9b3de6dcc6b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:33.345407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7ddd1b32-f372-4592-8d05-f9b3de6dcc6b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1314 microversion: 2.1 time: 0.345840 May 25 16:18:33.345984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 84/188] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:32 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1314 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:33.901714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-28d94864-e315-4ed4-beb6-f5443d8b1fca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:33.902318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-28d94864-e315-4ed4-beb6-f5443d8b1fca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:33.914418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28d94864-e315-4ed4-beb6-f5443d8b1fca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:33.921746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28d94864-e315-4ed4-beb6-f5443d8b1fca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:33.922874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28d94864-e315-4ed4-beb6-f5443d8b1fca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:33.925097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28d94864-e315-4ed4-beb6-f5443d8b1fca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:34.209189 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-28d94864-e315-4ed4-beb6-f5443d8b1fca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:34.233062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-28d94864-e315-4ed4-beb6-f5443d8b1fca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.335248 May 25 16:18:34.234224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 105/189] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:33 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:34.362046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e52d0aae-2f4e-4de4-9190-b7eb629fdd9c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:34.362761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e52d0aae-2f4e-4de4-9190-b7eb629fdd9c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:34.378894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e52d0aae-2f4e-4de4-9190-b7eb629fdd9c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:34.379360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e52d0aae-2f4e-4de4-9190-b7eb629fdd9c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:34.380121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e52d0aae-2f4e-4de4-9190-b7eb629fdd9c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:34.380540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e52d0aae-2f4e-4de4-9190-b7eb629fdd9c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:34.587893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e52d0aae-2f4e-4de4-9190-b7eb629fdd9c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:34.600807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e52d0aae-2f4e-4de4-9190-b7eb629fdd9c tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1498 microversion: 2.1 time: 0.241880 May 25 16:18:34.603637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 85/190] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:34 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1498 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:35.258663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ffc0a1cb-451e-4bde-be45-b2b31f1ec208 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:35.260183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ffc0a1cb-451e-4bde-be45-b2b31f1ec208 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:35.297470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ffc0a1cb-451e-4bde-be45-b2b31f1ec208 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:35.298171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ffc0a1cb-451e-4bde-be45-b2b31f1ec208 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:35.302915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ffc0a1cb-451e-4bde-be45-b2b31f1ec208 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:35.303424 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ffc0a1cb-451e-4bde-be45-b2b31f1ec208 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:35.465634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1328fa5b-11c9-41ba-8d6d-62564250207b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:35.466915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1328fa5b-11c9-41ba-8d6d-62564250207b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:35.482103 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1328fa5b-11c9-41ba-8d6d-62564250207b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:35.485157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1328fa5b-11c9-41ba-8d6d-62564250207b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:35.489433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1328fa5b-11c9-41ba-8d6d-62564250207b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:35.489884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1328fa5b-11c9-41ba-8d6d-62564250207b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:35.568157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1328fa5b-11c9-41ba-8d6d-62564250207b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Going to try to terminate instance {{(pid=117795) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:18:35.741759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1328fa5b-11c9-41ba-8d6d-62564250207b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "DELETE /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 204 len: 0 microversion: 2.1 time: 0.280808 May 25 16:18:35.747930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 86/191] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:18:35 2022] DELETE /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:18:35.768248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1faab93a-a7e4-4a34-8cd9-bda7f8179217 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:35.769678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1faab93a-a7e4-4a34-8cd9-bda7f8179217 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:35.779069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1faab93a-a7e4-4a34-8cd9-bda7f8179217 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:35.779952 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1faab93a-a7e4-4a34-8cd9-bda7f8179217 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:35.781152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1faab93a-a7e4-4a34-8cd9-bda7f8179217 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:35.781738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1faab93a-a7e4-4a34-8cd9-bda7f8179217 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:36.056745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ffc0a1cb-451e-4bde-be45-b2b31f1ec208 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:36.071670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ffc0a1cb-451e-4bde-be45-b2b31f1ec208 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.825090 May 25 16:18:36.085739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 106/192] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:35 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 840 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:36.243330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1faab93a-a7e4-4a34-8cd9-bda7f8179217 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:36.252184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1faab93a-a7e4-4a34-8cd9-bda7f8179217 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1504 microversion: 2.1 time: 0.502327 May 25 16:18:36.255290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 87/193] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:35 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1504 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:37.092680 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-073bae4d-3c5c-413a-bfc6-8e2e679e842a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:37.093363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-073bae4d-3c5c-413a-bfc6-8e2e679e842a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:37.113400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-073bae4d-3c5c-413a-bfc6-8e2e679e842a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:37.113974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-073bae4d-3c5c-413a-bfc6-8e2e679e842a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:37.114821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-073bae4d-3c5c-413a-bfc6-8e2e679e842a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:37.115291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-073bae4d-3c5c-413a-bfc6-8e2e679e842a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:37.273894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-283fe865-8188-45a1-9ec5-4d33e7167468 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:37.275207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-283fe865-8188-45a1-9ec5-4d33e7167468 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:37.282857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-283fe865-8188-45a1-9ec5-4d33e7167468 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:37.283662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-283fe865-8188-45a1-9ec5-4d33e7167468 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:37.284620 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-283fe865-8188-45a1-9ec5-4d33e7167468 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:37.285299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-283fe865-8188-45a1-9ec5-4d33e7167468 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:37.438114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-073bae4d-3c5c-413a-bfc6-8e2e679e842a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:37.448659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-073bae4d-3c5c-413a-bfc6-8e2e679e842a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.360100 May 25 16:18:37.449606 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 107/194] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:37 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:37.585899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-283fe865-8188-45a1-9ec5-4d33e7167468 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:37.599301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-283fe865-8188-45a1-9ec5-4d33e7167468 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1504 microversion: 2.1 time: 0.331260 May 25 16:18:37.600146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 88/195] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:37 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1504 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:38.474390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ab4474b7-a31c-4693-8ae2-4f134d664120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:38.475219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ab4474b7-a31c-4693-8ae2-4f134d664120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:38.484111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ab4474b7-a31c-4693-8ae2-4f134d664120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:38.484781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ab4474b7-a31c-4693-8ae2-4f134d664120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:38.486481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ab4474b7-a31c-4693-8ae2-4f134d664120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:38.487457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ab4474b7-a31c-4693-8ae2-4f134d664120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:38.744248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ab4474b7-a31c-4693-8ae2-4f134d664120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:38.760737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ab4474b7-a31c-4693-8ae2-4f134d664120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.298316 May 25 16:18:38.761754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 108/196] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:38 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:38.768219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-206ae227-a9a0-4003-986c-6dd762d49bde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:38.769132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-206ae227-a9a0-4003-986c-6dd762d49bde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:38.777832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-206ae227-a9a0-4003-986c-6dd762d49bde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:38.778434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-206ae227-a9a0-4003-986c-6dd762d49bde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:38.779411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-206ae227-a9a0-4003-986c-6dd762d49bde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:38.779959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-206ae227-a9a0-4003-986c-6dd762d49bde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:38.818506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-67d98ce2-82a4-43c0-846b-acc387fbf68d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Action: 'create', calling method: .version_select at 0x7f4a3249f9d0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-2139248485", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAOoPmNvFHukEdzVw/sNX9g6A1yGEwFryzQ/4IqDZUQMbbcdgPZp1xd911gubt+F01KYcC0HebqfdQzELaQOChPO1tndnhwLYU19u9bdFIlVw8Fdl1eXlQpwKqUVOS8GnA=="}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:38.833610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-67d98ce2-82a4-43c0-846b-acc387fbf68d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Getting quotas for project 01367a52e9104f85baec4b18ef8bf258. Resources: {'key_pairs'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:18:38.840450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-67d98ce2-82a4-43c0-846b-acc387fbf68d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Getting quotas for user 7da83fc4029c4cdebca07c7247669bd0 and project 01367a52e9104f85baec4b18ef8bf258. Resources: {'key_pairs'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:18:38.861274 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-206ae227-a9a0-4003-986c-6dd762d49bde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:38.876063 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-206ae227-a9a0-4003-986c-6dd762d49bde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 200 len: 1462 microversion: 2.1 time: 0.112911 May 25 16:18:38.877343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 109/197] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:38 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 1462 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:38.878866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-67d98ce2-82a4-43c0-846b-acc387fbf68d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.359219 May 25 16:18:38.880359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 89/198] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:18:38 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:38.882868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-ec723034-2418-4ff3-b836-8f21a7615df3 req-f6fe110e-303f-4d38-b74a-a9b27d407b5a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f674eae5-f61a-4079-b358-66339ea78ca8", "name": "network-vif-deleted", "tag": "bc347387-d953-4b27-aa60-a307fe381a78"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:38.892955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-ec723034-2418-4ff3-b836-8f21a7615df3 req-f6fe110e-303f-4d38-b74a-a9b27d407b5a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:38.893406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-ec723034-2418-4ff3-b836-8f21a7615df3 req-f6fe110e-303f-4d38-b74a-a9b27d407b5a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:38.905238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-ec723034-2418-4ff3-b836-8f21a7615df3 req-f6fe110e-303f-4d38-b74a-a9b27d407b5a service nova] Creating event network-vif-deleted:bc347387-d953-4b27-aa60-a307fe381a78 for instance f674eae5-f61a-4079-b358-66339ea78ca8 on hv-cinder-84327 May 25 16:18:38.910205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-ec723034-2418-4ff3-b836-8f21a7615df3 req-f6fe110e-303f-4d38-b74a-a9b27d407b5a service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031310 May 25 16:18:38.910867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 110/199] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:39.782934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0837601d-eef0-4bc7-8a1a-ab81f1c124d6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:39.783705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0837601d-eef0-4bc7-8a1a-ab81f1c124d6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:39.795851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0837601d-eef0-4bc7-8a1a-ab81f1c124d6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:39.797582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0837601d-eef0-4bc7-8a1a-ab81f1c124d6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:39.798906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0837601d-eef0-4bc7-8a1a-ab81f1c124d6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:39.799924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0837601d-eef0-4bc7-8a1a-ab81f1c124d6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:39.903525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:39.904153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: f674eae5-f61a-4079-b358-66339ea78ca8] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:39.918170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:39.919754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:39.921309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:39.922448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] HTTP exception thrown: Instance f674eae5-f61a-4079-b358-66339ea78ca8 could not be found. May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Request method failure captured: May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 HTTP/1.1 May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbI6til8OPEw8FGaflpQOyfgrZkd1x_7ZV4wFuridfr4B4Imdu6ldivOEa9T8DTXxRsqJGnEeU3c_CozdhfBut3Uab7rysYMBdf2Q30Oh_Dd1Mx4Kub-xYz5Pk8crkszn19XCiAwY80DGeEyL_e8QMHCOWa8PoSr3PPRzSe9YN6IOM May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 2aca09e1a3bc4926ba6c14f975728e39 May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesActionsTest-1820165728 May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:18:39.962068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_2aca09e1a3bc4926ba6c14f975728e39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/2aca09e1a3bc4926ba6c14f975728e39"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/2aca09e1a3bc4926ba6c14f975728e39"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/2aca09e1a3bc4926ba6c14f975728e39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesActionsTest-1820165728 May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 2aca09e1a3bc4926ba6c14f975728e39 May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesActionsTest-1820165728 May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesActionsTest-1820165728-project May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 63c237a15880463d9b3850dac6413ceb May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesActionsTest-1820165728-project May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance f674eae5-f61a-4079-b358-66339ea78ca8 could not be found. May 25 16:18:39.963804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'f674eae5-f61a-4079-b358-66339ea78ca8'} May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f674eae5-f61a-4079-b358-66339ea78ca8 could not be found. May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:18:39.964793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f674eae5-f61a-4079-b358-66339ea78ca8 could not be found. May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f674eae5-f61a-4079-b358-66339ea78ca8 could not be found. May 25 16:18:39.966322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:18:39.967475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Returning 404 to user: Instance f674eae5-f61a-4079-b358-66339ea78ca8 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:18:39.967475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b3b8a161-f58a-48c2-830d-1aa24654b904 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8" status: 404 len: 111 microversion: 2.1 time: 0.071373 May 25 16:18:39.974452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 111/200] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:39 2022] GET /compute/v2.1/servers/f674eae5-f61a-4079-b358-66339ea78ca8 => generated 111 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 25 16:18:40.018420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0837601d-eef0-4bc7-8a1a-ab81f1c124d6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:40.035080 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0837601d-eef0-4bc7-8a1a-ab81f1c124d6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.259252 May 25 16:18:40.041958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 90/201] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:39 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:40.125446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-317039479", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "networks": [{"uuid": "b13ad99f-3558-45d2-be4a-ca0dfa266747"}]}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:40.200088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.network.neutron [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] validate_networks() for [('b13ad99f-3558-45d2-be4a-ca0dfa266747', None, None, None, None, None)] {{(pid=117794) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:18:40.752322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Flavor limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:18:40.752982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Image limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:18:40.753732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Flavor pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:18:40.754448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Image pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:18:40.755025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:18:40.778463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:40.779209 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:40.780251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:40.780859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:40.839929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Getting quotas for project 2aca09e1a3bc4926ba6c14f975728e39. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:18:40.845032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Getting quotas for user 63c237a15880463d9b3850dac6413ceb and project 2aca09e1a3bc4926ba6c14f975728e39. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:18:40.857769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Going to run 1 instances... {{(pid=117794) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:18:40.860408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117794) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:18:40.951712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-25dc922d-a340-476d-8859-6dd7e6fe22be tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.829898 May 25 16:18:40.952914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 112/202] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:18:40 2022] POST /compute/v2.1/servers => generated 380 bytes in 832 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:18:40.963471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-06d266b2-97aa-4345-80d4-69e90e016df9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:40.964085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-06d266b2-97aa-4345-80d4-69e90e016df9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:41.037265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-06d266b2-97aa-4345-80d4-69e90e016df9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:41.045536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-06d266b2-97aa-4345-80d4-69e90e016df9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1213 microversion: 2.1 time: 0.086455 May 25 16:18:41.046777 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 91/203] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:40 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1213 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:41.059518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0d8dfa7a-fe0c-48a1-aa31-ef72d32578ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:41.060949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0d8dfa7a-fe0c-48a1-aa31-ef72d32578ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:41.070037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d8dfa7a-fe0c-48a1-aa31-ef72d32578ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:41.070758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d8dfa7a-fe0c-48a1-aa31-ef72d32578ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:41.071672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d8dfa7a-fe0c-48a1-aa31-ef72d32578ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:41.072111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d8dfa7a-fe0c-48a1-aa31-ef72d32578ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:41.267895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0d8dfa7a-fe0c-48a1-aa31-ef72d32578ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:41.284023 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0d8dfa7a-fe0c-48a1-aa31-ef72d32578ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.227785 May 25 16:18:41.285378 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 113/204] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:41 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:42.063478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cfb360b9-b6b2-4cb5-919f-70b760333ae7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:42.064171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cfb360b9-b6b2-4cb5-919f-70b760333ae7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:42.070926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cfb360b9-b6b2-4cb5-919f-70b760333ae7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:42.071464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cfb360b9-b6b2-4cb5-919f-70b760333ae7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:42.072306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cfb360b9-b6b2-4cb5-919f-70b760333ae7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:42.072818 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cfb360b9-b6b2-4cb5-919f-70b760333ae7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:42.135291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cfb360b9-b6b2-4cb5-919f-70b760333ae7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:42.143364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cfb360b9-b6b2-4cb5-919f-70b760333ae7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1265 microversion: 2.1 time: 0.083876 May 25 16:18:42.144203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 92/205] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:42 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1265 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:42.302393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-35a8ebd1-087c-4b5c-81e2-bfb1846bbdab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:42.303727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-35a8ebd1-087c-4b5c-81e2-bfb1846bbdab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:42.317750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35a8ebd1-087c-4b5c-81e2-bfb1846bbdab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:42.318275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35a8ebd1-087c-4b5c-81e2-bfb1846bbdab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:42.319136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35a8ebd1-087c-4b5c-81e2-bfb1846bbdab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:42.319972 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35a8ebd1-087c-4b5c-81e2-bfb1846bbdab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:42.579799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-35a8ebd1-087c-4b5c-81e2-bfb1846bbdab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:42.589523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-35a8ebd1-087c-4b5c-81e2-bfb1846bbdab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.291306 May 25 16:18:42.590103 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 114/206] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:42 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:43.160910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1126d909-38b9-4ee7-acca-40d8cce642f5 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:43.161516 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1126d909-38b9-4ee7-acca-40d8cce642f5 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:43.170070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1126d909-38b9-4ee7-acca-40d8cce642f5 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:43.170567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1126d909-38b9-4ee7-acca-40d8cce642f5 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:43.171472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1126d909-38b9-4ee7-acca-40d8cce642f5 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:43.171908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1126d909-38b9-4ee7-acca-40d8cce642f5 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:43.410451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1126d909-38b9-4ee7-acca-40d8cce642f5 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:43.419946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1126d909-38b9-4ee7-acca-40d8cce642f5 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.262695 May 25 16:18:43.433489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 93/207] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:43 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:43.606303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b35fdbe0-92a6-4ff0-8150-9442b9d3231e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:43.607922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b35fdbe0-92a6-4ff0-8150-9442b9d3231e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:43.621156 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b35fdbe0-92a6-4ff0-8150-9442b9d3231e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:43.621787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b35fdbe0-92a6-4ff0-8150-9442b9d3231e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:43.622651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b35fdbe0-92a6-4ff0-8150-9442b9d3231e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:43.623185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b35fdbe0-92a6-4ff0-8150-9442b9d3231e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:43.920120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b35fdbe0-92a6-4ff0-8150-9442b9d3231e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:43.930036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b35fdbe0-92a6-4ff0-8150-9442b9d3231e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.327971 May 25 16:18:43.930933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 115/208] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:43 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:44.183539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-08e35893-eb29-4dcd-8fa9-5219f8db8565 req-8befb518-fb34-464d-94e5-914e30c52077 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "79123bd5-06c0-402b-86fe-3068c15d1a8d", "tag": "37ce84d7-52f5-4f34-97cc-720adf4c401b"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:44.192964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-08e35893-eb29-4dcd-8fa9-5219f8db8565 req-8befb518-fb34-464d-94e5-914e30c52077 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:44.193782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-08e35893-eb29-4dcd-8fa9-5219f8db8565 req-8befb518-fb34-464d-94e5-914e30c52077 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:44.206128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-08e35893-eb29-4dcd-8fa9-5219f8db8565 req-8befb518-fb34-464d-94e5-914e30c52077 service nova] Creating event network-changed:37ce84d7-52f5-4f34-97cc-720adf4c401b for instance 79123bd5-06c0-402b-86fe-3068c15d1a8d on hv-cinder-84327 May 25 16:18:44.211893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-08e35893-eb29-4dcd-8fa9-5219f8db8565 req-8befb518-fb34-464d-94e5-914e30c52077 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040631 May 25 16:18:44.212628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 94/209] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:18:44.438492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4a7abf8e-c4d2-411d-8b78-8982286c4dde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:44.442846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4a7abf8e-c4d2-411d-8b78-8982286c4dde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:44.446952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=3 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:44.451474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4a7abf8e-c4d2-411d-8b78-8982286c4dde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:44.451992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4a7abf8e-c4d2-411d-8b78-8982286c4dde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:44.452958 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4a7abf8e-c4d2-411d-8b78-8982286c4dde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:44.453552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4a7abf8e-c4d2-411d-8b78-8982286c4dde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:44.459219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:44.670695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4a7abf8e-c4d2-411d-8b78-8982286c4dde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:44.684531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4a7abf8e-c4d2-411d-8b78-8982286c4dde tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.250277 May 25 16:18:44.685662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 116/210] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:44 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:44.947563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3e235337-03c1-4869-a039-1aa6efd69ca6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:44.948540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3e235337-03c1-4869-a039-1aa6efd69ca6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:44.955530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e235337-03c1-4869-a039-1aa6efd69ca6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:44.955982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e235337-03c1-4869-a039-1aa6efd69ca6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:44.956751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e235337-03c1-4869-a039-1aa6efd69ca6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:44.957182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e235337-03c1-4869-a039-1aa6efd69ca6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:45.222713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3e235337-03c1-4869-a039-1aa6efd69ca6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:45.236078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3e235337-03c1-4869-a039-1aa6efd69ca6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.292077 May 25 16:18:45.240903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 95/211] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:44 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:45.704994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ff829cf6-33f5-4627-b55b-eb873b4d4223 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:45.705758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ff829cf6-33f5-4627-b55b-eb873b4d4223 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:45.712409 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff829cf6-33f5-4627-b55b-eb873b4d4223 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:45.712866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff829cf6-33f5-4627-b55b-eb873b4d4223 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:45.713626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff829cf6-33f5-4627-b55b-eb873b4d4223 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:45.714068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff829cf6-33f5-4627-b55b-eb873b4d4223 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:45.856904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ff829cf6-33f5-4627-b55b-eb873b4d4223 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:45.866803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ff829cf6-33f5-4627-b55b-eb873b4d4223 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.165551 May 25 16:18:45.867441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 117/212] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:45 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:46.254937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-73594a89-d54d-4930-b8cb-53b30b675262 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:46.255549 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-73594a89-d54d-4930-b8cb-53b30b675262 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:46.262191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-73594a89-d54d-4930-b8cb-53b30b675262 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:46.263251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-73594a89-d54d-4930-b8cb-53b30b675262 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:46.264238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-73594a89-d54d-4930-b8cb-53b30b675262 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:46.264988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-73594a89-d54d-4930-b8cb-53b30b675262 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:46.404171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-73594a89-d54d-4930-b8cb-53b30b675262 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:46.412008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-73594a89-d54d-4930-b8cb-53b30b675262 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.161008 May 25 16:18:46.412568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 96/213] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:46 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:46.884729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-456d5553-c8a3-4af4-abf3-cf21df84990a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:46.886368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-456d5553-c8a3-4af4-abf3-cf21df84990a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:46.893595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-456d5553-c8a3-4af4-abf3-cf21df84990a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:46.893707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-456d5553-c8a3-4af4-abf3-cf21df84990a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:46.894567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-456d5553-c8a3-4af4-abf3-cf21df84990a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:46.894964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-456d5553-c8a3-4af4-abf3-cf21df84990a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:47.030704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-456d5553-c8a3-4af4-abf3-cf21df84990a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:47.039073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-456d5553-c8a3-4af4-abf3-cf21df84990a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.158563 May 25 16:18:47.039979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 118/214] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:46 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:47.430196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6e3c5cc6-e800-4502-b205-6b448266b46a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:47.430769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6e3c5cc6-e800-4502-b205-6b448266b46a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:47.437336 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e3c5cc6-e800-4502-b205-6b448266b46a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:47.437790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e3c5cc6-e800-4502-b205-6b448266b46a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:47.438541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e3c5cc6-e800-4502-b205-6b448266b46a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:47.438969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e3c5cc6-e800-4502-b205-6b448266b46a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:47.579562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6e3c5cc6-e800-4502-b205-6b448266b46a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:47.589268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6e3c5cc6-e800-4502-b205-6b448266b46a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.163155 May 25 16:18:47.590162 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 97/215] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:47 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:48.056529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0eb5fce2-0f25-4cd6-a399-55f2bc9e3784 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:48.057132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0eb5fce2-0f25-4cd6-a399-55f2bc9e3784 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:48.064179 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eb5fce2-0f25-4cd6-a399-55f2bc9e3784 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:48.064679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eb5fce2-0f25-4cd6-a399-55f2bc9e3784 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:48.065503 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eb5fce2-0f25-4cd6-a399-55f2bc9e3784 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:48.065974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eb5fce2-0f25-4cd6-a399-55f2bc9e3784 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:48.210056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0eb5fce2-0f25-4cd6-a399-55f2bc9e3784 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:48.218627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0eb5fce2-0f25-4cd6-a399-55f2bc9e3784 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.166052 May 25 16:18:48.219228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 119/216] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:48 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:48.610407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fb44cf6e-879d-4e25-9182-b412c5a6d1df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:48.611535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fb44cf6e-879d-4e25-9182-b412c5a6d1df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:48.620039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb44cf6e-879d-4e25-9182-b412c5a6d1df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:48.620690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb44cf6e-879d-4e25-9182-b412c5a6d1df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:48.621536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb44cf6e-879d-4e25-9182-b412c5a6d1df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:48.622053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb44cf6e-879d-4e25-9182-b412c5a6d1df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:48.777110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fb44cf6e-879d-4e25-9182-b412c5a6d1df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:48.785608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fb44cf6e-879d-4e25-9182-b412c5a6d1df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.179713 May 25 16:18:48.786387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 98/217] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:48 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:49.236674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-918ec883-7b55-480e-a62d-a09d985498d9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:49.237326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-918ec883-7b55-480e-a62d-a09d985498d9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:49.244513 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-918ec883-7b55-480e-a62d-a09d985498d9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:49.244991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-918ec883-7b55-480e-a62d-a09d985498d9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:49.245809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-918ec883-7b55-480e-a62d-a09d985498d9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:49.246251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-918ec883-7b55-480e-a62d-a09d985498d9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:49.385703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-918ec883-7b55-480e-a62d-a09d985498d9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:49.394343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-918ec883-7b55-480e-a62d-a09d985498d9 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.161867 May 25 16:18:49.394928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 120/218] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:49 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:49.804101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c14752fc-2c13-4091-bb03-cb4ee11f837f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:49.804696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c14752fc-2c13-4091-bb03-cb4ee11f837f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:49.811736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c14752fc-2c13-4091-bb03-cb4ee11f837f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:49.812186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c14752fc-2c13-4091-bb03-cb4ee11f837f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:49.812959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c14752fc-2c13-4091-bb03-cb4ee11f837f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:49.813392 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c14752fc-2c13-4091-bb03-cb4ee11f837f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:49.950950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c14752fc-2c13-4091-bb03-cb4ee11f837f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:49.953623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:49.963108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c14752fc-2c13-4091-bb03-cb4ee11f837f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.162977 May 25 16:18:49.963913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 99/219] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:49 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:50.413324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-03cf3ac3-3f43-4084-9d2f-349b3526cb1a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:50.414045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-03cf3ac3-3f43-4084-9d2f-349b3526cb1a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:50.421808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-03cf3ac3-3f43-4084-9d2f-349b3526cb1a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:50.422257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-03cf3ac3-3f43-4084-9d2f-349b3526cb1a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:50.423057 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-03cf3ac3-3f43-4084-9d2f-349b3526cb1a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:50.423548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-03cf3ac3-3f43-4084-9d2f-349b3526cb1a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:50.575456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-03cf3ac3-3f43-4084-9d2f-349b3526cb1a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:50.583867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-03cf3ac3-3f43-4084-9d2f-349b3526cb1a tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.175712 May 25 16:18:50.584479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 121/220] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:50 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:50.984730 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6c080f93-21f7-4947-8f9e-ce727a6f273c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:50.985378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6c080f93-21f7-4947-8f9e-ce727a6f273c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:50.993561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c080f93-21f7-4947-8f9e-ce727a6f273c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:50.993923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c080f93-21f7-4947-8f9e-ce727a6f273c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:50.994693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c080f93-21f7-4947-8f9e-ce727a6f273c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:50.995123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c080f93-21f7-4947-8f9e-ce727a6f273c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:51.045575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=6 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:51.150520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6c080f93-21f7-4947-8f9e-ce727a6f273c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:51.158952 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6c080f93-21f7-4947-8f9e-ce727a6f273c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.180520 May 25 16:18:51.160005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 100/221] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:50 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:51.605785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-826d739f-dd36-4551-b1cd-291f23c46db7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:51.606453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-826d739f-dd36-4551-b1cd-291f23c46db7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:51.608805 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:51.609578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=6 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:51.628593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-826d739f-dd36-4551-b1cd-291f23c46db7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:51.629107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-826d739f-dd36-4551-b1cd-291f23c46db7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:51.629937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-826d739f-dd36-4551-b1cd-291f23c46db7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:51.635301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-826d739f-dd36-4551-b1cd-291f23c46db7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:51.775812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-826d739f-dd36-4551-b1cd-291f23c46db7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:51.786232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-826d739f-dd36-4551-b1cd-291f23c46db7 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.189415 May 25 16:18:51.786946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 122/222] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:51 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:52.177403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d546dc2e-4c13-4637-b1a6-fb095017b312 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:52.178106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d546dc2e-4c13-4637-b1a6-fb095017b312 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:52.185107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d546dc2e-4c13-4637-b1a6-fb095017b312 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:52.185623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d546dc2e-4c13-4637-b1a6-fb095017b312 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:52.186456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d546dc2e-4c13-4637-b1a6-fb095017b312 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:52.186922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d546dc2e-4c13-4637-b1a6-fb095017b312 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:52.335886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d546dc2e-4c13-4637-b1a6-fb095017b312 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:52.344362 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d546dc2e-4c13-4637-b1a6-fb095017b312 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.171311 May 25 16:18:52.345195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 101/223] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:52 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:52.805504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-accaf2e8-ccf7-4b61-b398-4a285aea1edd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:52.806134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-accaf2e8-ccf7-4b61-b398-4a285aea1edd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:52.813500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-accaf2e8-ccf7-4b61-b398-4a285aea1edd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:52.813966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-accaf2e8-ccf7-4b61-b398-4a285aea1edd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:52.814727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-accaf2e8-ccf7-4b61-b398-4a285aea1edd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:52.815186 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-accaf2e8-ccf7-4b61-b398-4a285aea1edd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:52.957135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-accaf2e8-ccf7-4b61-b398-4a285aea1edd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:52.965970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-accaf2e8-ccf7-4b61-b398-4a285aea1edd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.164684 May 25 16:18:52.966679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 123/224] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:52 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:53.363087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f5601c83-aefe-4d54-97ea-a906393c716f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:53.363805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f5601c83-aefe-4d54-97ea-a906393c716f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:53.370954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5601c83-aefe-4d54-97ea-a906393c716f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:53.371846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5601c83-aefe-4d54-97ea-a906393c716f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:53.372708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5601c83-aefe-4d54-97ea-a906393c716f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:53.373153 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5601c83-aefe-4d54-97ea-a906393c716f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:53.537932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f5601c83-aefe-4d54-97ea-a906393c716f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:53.551625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f5601c83-aefe-4d54-97ea-a906393c716f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.193064 May 25 16:18:53.563313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 102/225] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:53 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:53.985891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-11cfd299-e11f-4fc2-945b-eccc684264bd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:53.987012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-11cfd299-e11f-4fc2-945b-eccc684264bd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:53.998698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11cfd299-e11f-4fc2-945b-eccc684264bd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:53.999185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11cfd299-e11f-4fc2-945b-eccc684264bd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:54.000489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11cfd299-e11f-4fc2-945b-eccc684264bd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:54.001097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11cfd299-e11f-4fc2-945b-eccc684264bd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:54.244593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-11cfd299-e11f-4fc2-945b-eccc684264bd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:54.252422 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-11cfd299-e11f-4fc2-945b-eccc684264bd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.272631 May 25 16:18:54.253040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 124/226] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:53 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:54.568825 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-220a5e34-ac13-4aaa-a657-7912178ba08c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:54.570182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-220a5e34-ac13-4aaa-a657-7912178ba08c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:54.576823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-220a5e34-ac13-4aaa-a657-7912178ba08c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:54.577251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-220a5e34-ac13-4aaa-a657-7912178ba08c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:54.578023 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-220a5e34-ac13-4aaa-a657-7912178ba08c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:54.578457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-220a5e34-ac13-4aaa-a657-7912178ba08c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:54.790690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-220a5e34-ac13-4aaa-a657-7912178ba08c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:54.799076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-220a5e34-ac13-4aaa-a657-7912178ba08c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.234311 May 25 16:18:54.799906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 103/227] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:54 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:55.270338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f3bd9ba5-b3ff-40ab-a3fa-6e8fb9f21717 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:55.271711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f3bd9ba5-b3ff-40ab-a3fa-6e8fb9f21717 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:55.279207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f3bd9ba5-b3ff-40ab-a3fa-6e8fb9f21717 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:55.280044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f3bd9ba5-b3ff-40ab-a3fa-6e8fb9f21717 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:55.280981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f3bd9ba5-b3ff-40ab-a3fa-6e8fb9f21717 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:55.281552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f3bd9ba5-b3ff-40ab-a3fa-6e8fb9f21717 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:55.591003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f3bd9ba5-b3ff-40ab-a3fa-6e8fb9f21717 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:55.606599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f3bd9ba5-b3ff-40ab-a3fa-6e8fb9f21717 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.340666 May 25 16:18:55.607227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 125/228] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:55 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:55.817211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7a1012db-b12c-449e-a118-05ef587699fd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:55.817861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7a1012db-b12c-449e-a118-05ef587699fd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:55.830331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1012db-b12c-449e-a118-05ef587699fd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:55.830891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1012db-b12c-449e-a118-05ef587699fd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:55.831863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1012db-b12c-449e-a118-05ef587699fd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:55.835585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1012db-b12c-449e-a118-05ef587699fd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:56.357194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7a1012db-b12c-449e-a118-05ef587699fd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:56.366241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7a1012db-b12c-449e-a118-05ef587699fd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1522 microversion: 2.1 time: 0.553655 May 25 16:18:56.376475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 104/229] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:55 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1522 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:56.624070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c3ac4999-000d-4b29-b4ec-40f0a1263670 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:56.624983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c3ac4999-000d-4b29-b4ec-40f0a1263670 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:56.632810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3ac4999-000d-4b29-b4ec-40f0a1263670 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:56.634056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3ac4999-000d-4b29-b4ec-40f0a1263670 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:56.634908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3ac4999-000d-4b29-b4ec-40f0a1263670 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:56.635535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3ac4999-000d-4b29-b4ec-40f0a1263670 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:56.835222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c3ac4999-000d-4b29-b4ec-40f0a1263670 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:56.846729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c3ac4999-000d-4b29-b4ec-40f0a1263670 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.226821 May 25 16:18:56.847990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 126/230] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:56 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:57.136304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-aa2e95ad-910b-41c8-96f1-476a4a8c2690 req-52683feb-bfd1-4ef5-a13c-d4a00119295c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "79123bd5-06c0-402b-86fe-3068c15d1a8d", "name": "network-vif-plugged", "status": "completed", "tag": "37ce84d7-52f5-4f34-97cc-720adf4c401b"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:57.151705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-aa2e95ad-910b-41c8-96f1-476a4a8c2690 req-52683feb-bfd1-4ef5-a13c-d4a00119295c service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:57.152089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-aa2e95ad-910b-41c8-96f1-476a4a8c2690 req-52683feb-bfd1-4ef5-a13c-d4a00119295c service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:57.184164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-aa2e95ad-910b-41c8-96f1-476a4a8c2690 req-52683feb-bfd1-4ef5-a13c-d4a00119295c service nova] Creating event network-vif-plugged:37ce84d7-52f5-4f34-97cc-720adf4c401b for instance 79123bd5-06c0-402b-86fe-3068c15d1a8d on hv-cinder-84327 May 25 16:18:57.197513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-aa2e95ad-910b-41c8-96f1-476a4a8c2690 req-52683feb-bfd1-4ef5-a13c-d4a00119295c service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065635 May 25 16:18:57.207115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 105/231] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:18:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:18:57.387408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-261e0220-b62a-46c1-bbdd-ed2c619e112e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:57.392046 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-261e0220-b62a-46c1-bbdd-ed2c619e112e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:57.415447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-261e0220-b62a-46c1-bbdd-ed2c619e112e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:57.415838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-261e0220-b62a-46c1-bbdd-ed2c619e112e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:57.417391 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-261e0220-b62a-46c1-bbdd-ed2c619e112e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:57.417992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-261e0220-b62a-46c1-bbdd-ed2c619e112e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:57.651656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-261e0220-b62a-46c1-bbdd-ed2c619e112e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:57.661235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-261e0220-b62a-46c1-bbdd-ed2c619e112e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1502 microversion: 2.1 time: 0.281651 May 25 16:18:57.662436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 127/232] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:57 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1502 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:57.678928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7e9c32c1-093a-44c5-beb5-54ddc727a2a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:57.679884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7e9c32c1-093a-44c5-beb5-54ddc727a2a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:57.687399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e9c32c1-093a-44c5-beb5-54ddc727a2a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:57.688210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e9c32c1-093a-44c5-beb5-54ddc727a2a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:57.688989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e9c32c1-093a-44c5-beb5-54ddc727a2a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:57.689423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e9c32c1-093a-44c5-beb5-54ddc727a2a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:57.863228 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7e9c32c1-093a-44c5-beb5-54ddc727a2a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:57.866031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-34beb798-0094-45bd-b6bc-544e857334bc tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:57.866642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-34beb798-0094-45bd-b6bc-544e857334bc tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:57.874233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34beb798-0094-45bd-b6bc-544e857334bc tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:57.874675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34beb798-0094-45bd-b6bc-544e857334bc tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:57.875234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7e9c32c1-093a-44c5-beb5-54ddc727a2a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1514 microversion: 2.1 time: 0.200212 May 25 16:18:57.875943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34beb798-0094-45bd-b6bc-544e857334bc tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:57.876370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34beb798-0094-45bd-b6bc-544e857334bc tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:57.888133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 106/233] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:57 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1514 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:58.095068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-34beb798-0094-45bd-b6bc-544e857334bc tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:58.104504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-34beb798-0094-45bd-b6bc-544e857334bc tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.243368 May 25 16:18:58.106208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 128/234] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:57 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:18:59.122354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8b950b4a-4e7f-46da-9962-2ad8a6e19387 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:59.123388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8b950b4a-4e7f-46da-9962-2ad8a6e19387 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:59.130183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b950b4a-4e7f-46da-9962-2ad8a6e19387 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:59.130781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b950b4a-4e7f-46da-9962-2ad8a6e19387 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:59.131706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b950b4a-4e7f-46da-9962-2ad8a6e19387 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:59.132256 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b950b4a-4e7f-46da-9962-2ad8a6e19387 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:59.289010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8b950b4a-4e7f-46da-9962-2ad8a6e19387 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:59.300438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8b950b4a-4e7f-46da-9962-2ad8a6e19387 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1313 microversion: 2.1 time: 0.182099 May 25 16:18:59.301210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 107/235] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:59 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1313 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:18:59.396975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-55506c89-3927-4b9b-90d1-46fbffb6ef2b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:18:59.398759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-55506c89-3927-4b9b-90d1-46fbffb6ef2b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:59.407570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55506c89-3927-4b9b-90d1-46fbffb6ef2b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:59.408218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55506c89-3927-4b9b-90d1-46fbffb6ef2b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:59.409510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55506c89-3927-4b9b-90d1-46fbffb6ef2b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:59.410241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55506c89-3927-4b9b-90d1-46fbffb6ef2b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:59.439068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-55506c89-3927-4b9b-90d1-46fbffb6ef2b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lazy-loading 'flavor' on Instance uuid 69fc8c5b-82f0-4a03-8dab-381fdee5884d {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:18:59.558907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-55506c89-3927-4b9b-90d1-46fbffb6ef2b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d/action" status: 200 len: 32 microversion: 2.1 time: 0.166072 May 25 16:18:59.563585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 129/236] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:18:59 2022] POST /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d/action => generated 32 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 25 16:18:59.578485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-86c97886-c9bb-47c8-8987-307840475b73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:18:59.583770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-86c97886-c9bb-47c8-8987-307840475b73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:18:59.591310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86c97886-c9bb-47c8-8987-307840475b73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:59.592434 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86c97886-c9bb-47c8-8987-307840475b73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:59.594275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86c97886-c9bb-47c8-8987-307840475b73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:18:59.594908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86c97886-c9bb-47c8-8987-307840475b73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:18:59.974643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-86c97886-c9bb-47c8-8987-307840475b73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:18:59.986683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-86c97886-c9bb-47c8-8987-307840475b73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.419284 May 25 16:18:59.989956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 108/237] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:18:59 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:00.324978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-64118ff7-74a2-4cdf-9511-11309c79324f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:00.326068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-64118ff7-74a2-4cdf-9511-11309c79324f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:00.333701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64118ff7-74a2-4cdf-9511-11309c79324f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:00.334455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64118ff7-74a2-4cdf-9511-11309c79324f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:00.335470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64118ff7-74a2-4cdf-9511-11309c79324f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:00.336325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64118ff7-74a2-4cdf-9511-11309c79324f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:00.514375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-64118ff7-74a2-4cdf-9511-11309c79324f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:00.521989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:00.527044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-64118ff7-74a2-4cdf-9511-11309c79324f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1497 microversion: 2.1 time: 0.211686 May 25 16:19:00.540266 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 130/238] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:00 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1497 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:01.010802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7cfa13c0-c74b-4774-93b0-025a7388fd08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:01.011461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7cfa13c0-c74b-4774-93b0-025a7388fd08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:01.018683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7cfa13c0-c74b-4774-93b0-025a7388fd08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:01.019394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7cfa13c0-c74b-4774-93b0-025a7388fd08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:01.030733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7cfa13c0-c74b-4774-93b0-025a7388fd08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:01.031447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7cfa13c0-c74b-4774-93b0-025a7388fd08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:01.303010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7cfa13c0-c74b-4774-93b0-025a7388fd08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:01.313869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7cfa13c0-c74b-4774-93b0-025a7388fd08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.312166 May 25 16:19:01.315839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 109/239] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:01 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:02.333946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9af592ff-fc6c-4521-8aad-2a8bc0ec595d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:02.335722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9af592ff-fc6c-4521-8aad-2a8bc0ec595d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:02.356477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9af592ff-fc6c-4521-8aad-2a8bc0ec595d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:02.357251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9af592ff-fc6c-4521-8aad-2a8bc0ec595d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:02.358238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9af592ff-fc6c-4521-8aad-2a8bc0ec595d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:02.358870 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9af592ff-fc6c-4521-8aad-2a8bc0ec595d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:02.619540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9af592ff-fc6c-4521-8aad-2a8bc0ec595d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:02.635806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9af592ff-fc6c-4521-8aad-2a8bc0ec595d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.307401 May 25 16:19:02.636738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 131/240] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:02 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:03.660522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9e8a0077-b4bf-4d80-8b80-7a27a0d9f15f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:03.661923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9e8a0077-b4bf-4d80-8b80-7a27a0d9f15f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:03.669767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=68,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:03.692351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e8a0077-b4bf-4d80-8b80-7a27a0d9f15f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:03.693578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e8a0077-b4bf-4d80-8b80-7a27a0d9f15f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:03.694460 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e8a0077-b4bf-4d80-8b80-7a27a0d9f15f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:03.694978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e8a0077-b4bf-4d80-8b80-7a27a0d9f15f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:03.878771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9e8a0077-b4bf-4d80-8b80-7a27a0d9f15f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:03.888949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9e8a0077-b4bf-4d80-8b80-7a27a0d9f15f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.232489 May 25 16:19:03.889547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 110/241] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:03 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:04.001579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a5c3db06-c307-4617-be54-9911fb635888 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:04.002091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a5c3db06-c307-4617-be54-9911fb635888 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:04.009188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5c3db06-c307-4617-be54-9911fb635888 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:04.009646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5c3db06-c307-4617-be54-9911fb635888 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:04.010414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5c3db06-c307-4617-be54-9911fb635888 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:04.011048 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5c3db06-c307-4617-be54-9911fb635888 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:04.042881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a5c3db06-c307-4617-be54-9911fb635888 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:19:04.175288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a5c3db06-c307-4617-be54-9911fb635888 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "DELETE /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 204 len: 0 microversion: 2.1 time: 0.176363 May 25 16:19:04.175909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 132/242] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:19:03 2022] DELETE /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:19:04.188588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-49c880b8-3df0-44a3-919a-f810fa3bc890 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:04.189285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-49c880b8-3df0-44a3-919a-f810fa3bc890 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:04.208261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49c880b8-3df0-44a3-919a-f810fa3bc890 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:04.210076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49c880b8-3df0-44a3-919a-f810fa3bc890 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:04.214686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49c880b8-3df0-44a3-919a-f810fa3bc890 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:04.217055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49c880b8-3df0-44a3-919a-f810fa3bc890 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:04.453442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-49c880b8-3df0-44a3-919a-f810fa3bc890 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:04.470757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-49c880b8-3df0-44a3-919a-f810fa3bc890 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1503 microversion: 2.1 time: 0.290161 May 25 16:19:04.473294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 111/243] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:04 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1503 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:04.909260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c167afe1-0e9c-4677-9330-072980c1f2b6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:04.910567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c167afe1-0e9c-4677-9330-072980c1f2b6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:04.923779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c167afe1-0e9c-4677-9330-072980c1f2b6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:04.925480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c167afe1-0e9c-4677-9330-072980c1f2b6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:04.926640 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c167afe1-0e9c-4677-9330-072980c1f2b6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:04.927867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c167afe1-0e9c-4677-9330-072980c1f2b6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:05.121540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c167afe1-0e9c-4677-9330-072980c1f2b6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:05.129882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c167afe1-0e9c-4677-9330-072980c1f2b6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.226665 May 25 16:19:05.131136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 133/244] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:04 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:05.169677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e85e4d53-45d3-4ac6-826f-c5b524fcbf81 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:05.175708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e85e4d53-45d3-4ac6-826f-c5b524fcbf81 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:05.183234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e85e4d53-45d3-4ac6-826f-c5b524fcbf81 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:05.183946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e85e4d53-45d3-4ac6-826f-c5b524fcbf81 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:05.184928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e85e4d53-45d3-4ac6-826f-c5b524fcbf81 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:05.185516 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e85e4d53-45d3-4ac6-826f-c5b524fcbf81 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:05.214650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e85e4d53-45d3-4ac6-826f-c5b524fcbf81 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/ips" status: 200 len: 146 microversion: 2.1 time: 0.049821 May 25 16:19:05.215437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 112/245] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:19:05 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/ips => generated 146 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:05.488521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ca55abba-901c-4d18-9365-a0db2cb7f6cd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:05.489313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ca55abba-901c-4d18-9365-a0db2cb7f6cd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:05.500185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca55abba-901c-4d18-9365-a0db2cb7f6cd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:05.502241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca55abba-901c-4d18-9365-a0db2cb7f6cd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:05.503051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca55abba-901c-4d18-9365-a0db2cb7f6cd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:05.504492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca55abba-901c-4d18-9365-a0db2cb7f6cd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:05.737047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ca55abba-901c-4d18-9365-a0db2cb7f6cd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:05.753992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ca55abba-901c-4d18-9365-a0db2cb7f6cd tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1503 microversion: 2.1 time: 0.268978 May 25 16:19:05.755109 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 134/246] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:05 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1503 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:05.820705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-900316929", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-2139248485", "security_groups": [{"name": "tempest-secgroup-smoke-170858155"}], "block_device_mapping_v2": [{"uuid": "b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "7fd1aa0d-38cb-4e70-ab28-67c3630862cc"}]}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:06.012783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] validate_networks() for [('7fd1aa0d-38cb-4e70-ab28-67c3630862cc', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:19:06.147101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d89038b5-8ad0-45a6-899e-fbd288621e84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:06.148040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d89038b5-8ad0-45a6-899e-fbd288621e84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:06.160738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d89038b5-8ad0-45a6-899e-fbd288621e84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:06.161197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d89038b5-8ad0-45a6-899e-fbd288621e84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:06.162100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d89038b5-8ad0-45a6-899e-fbd288621e84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:06.162490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d89038b5-8ad0-45a6-899e-fbd288621e84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:06.673730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d89038b5-8ad0-45a6-899e-fbd288621e84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:06.696082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d89038b5-8ad0-45a6-899e-fbd288621e84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.542009 May 25 16:19:06.701311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 135/247] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:06 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:06.774820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f50a5002-97a3-4b76-a534-bbf056b8801f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:06.775623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f50a5002-97a3-4b76-a534-bbf056b8801f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:06.800072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f50a5002-97a3-4b76-a534-bbf056b8801f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:06.800931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f50a5002-97a3-4b76-a534-bbf056b8801f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:06.802303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f50a5002-97a3-4b76-a534-bbf056b8801f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:06.804037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f50a5002-97a3-4b76-a534-bbf056b8801f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:07.202394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f50a5002-97a3-4b76-a534-bbf056b8801f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:07.239064 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f50a5002-97a3-4b76-a534-bbf056b8801f tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1503 microversion: 2.1 time: 0.468909 May 25 16:19:07.240463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 136/248] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:06 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1503 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:07.716014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cb774d2d-f643-4915-8c2c-0c2993c81601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:07.717145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cb774d2d-f643-4915-8c2c-0c2993c81601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:07.729741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb774d2d-f643-4915-8c2c-0c2993c81601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:07.730584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb774d2d-f643-4915-8c2c-0c2993c81601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:07.735568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb774d2d-f643-4915-8c2c-0c2993c81601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:07.736565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb774d2d-f643-4915-8c2c-0c2993c81601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:07.782785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:19:07.784186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:19:07.784991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:19:07.785567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:19:07.786139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:19:07.801807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:07.802511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:07.804555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:07.805674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:07.869361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Getting quotas for project 01367a52e9104f85baec4b18ef8bf258. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:19:07.883635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Getting quotas for user 7da83fc4029c4cdebca07c7247669bd0 and project 01367a52e9104f85baec4b18ef8bf258. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:19:08.031104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:19:08.032722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79',volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:19:08.201382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cb774d2d-f643-4915-8c2c-0c2993c81601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:08.209956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cb774d2d-f643-4915-8c2c-0c2993c81601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.498461 May 25 16:19:08.224583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 137/249] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:07 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:08.257636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6cdc8bdf-9ed8-4348-82fc-c5e43ad0b2d1 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:08.258308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6cdc8bdf-9ed8-4348-82fc-c5e43ad0b2d1 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:08.274560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-976df105-2bd7-46fe-9fa8-b93068b16103 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 2.457276 May 25 16:19:08.276289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 113/250] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:19:05 2022] POST /compute/v2.1/servers => generated 405 bytes in 2460 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:19:08.283601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cdc8bdf-9ed8-4348-82fc-c5e43ad0b2d1 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:08.284364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cdc8bdf-9ed8-4348-82fc-c5e43ad0b2d1 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:08.285358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cdc8bdf-9ed8-4348-82fc-c5e43ad0b2d1 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:08.285981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cdc8bdf-9ed8-4348-82fc-c5e43ad0b2d1 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:08.298794 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-af9e1752-b841-4777-932d-6ec990ec12d8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:08.300628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-af9e1752-b841-4777-932d-6ec990ec12d8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:08.479638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-af9e1752-b841-4777-932d-6ec990ec12d8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:08.487101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-af9e1752-b841-4777-932d-6ec990ec12d8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1094 microversion: 2.1 time: 0.203116 May 25 16:19:08.488517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 114/251] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:08 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1094 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:08.518236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6cdc8bdf-9ed8-4348-82fc-c5e43ad0b2d1 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '63c237a15880463d9b3850dac6413ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2aca09e1a3bc4926ba6c14f975728e39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:08.530114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6cdc8bdf-9ed8-4348-82fc-c5e43ad0b2d1 tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 200 len: 1461 microversion: 2.1 time: 0.276894 May 25 16:19:08.543313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 138/252] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:08 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 1461 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:08.569385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-2e76c3be-f964-4471-bdbf-7c527a8723fa req-2fc69516-f55a-45d1-902d-085dc2a4bbfc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "79123bd5-06c0-402b-86fe-3068c15d1a8d", "name": "network-vif-deleted", "tag": "37ce84d7-52f5-4f34-97cc-720adf4c401b"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:08.592284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-2e76c3be-f964-4471-bdbf-7c527a8723fa req-2fc69516-f55a-45d1-902d-085dc2a4bbfc service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:08.592740 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-2e76c3be-f964-4471-bdbf-7c527a8723fa req-2fc69516-f55a-45d1-902d-085dc2a4bbfc service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:08.620886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-2e76c3be-f964-4471-bdbf-7c527a8723fa req-2fc69516-f55a-45d1-902d-085dc2a4bbfc service nova] Creating event network-vif-deleted:37ce84d7-52f5-4f34-97cc-720adf4c401b for instance 79123bd5-06c0-402b-86fe-3068c15d1a8d on hv-cinder-84327 May 25 16:19:08.722604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-2e76c3be-f964-4471-bdbf-7c527a8723fa req-2fc69516-f55a-45d1-902d-085dc2a4bbfc service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.156720 May 25 16:19:08.731872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 115/253] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:09.034309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a1031b65-e30e-450c-bd57-4d3a36992072 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b546e80a-6961-4be5-8ae2-198ede1fbbf9"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:09.036763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a1031b65-e30e-450c-bd57-4d3a36992072 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:09.047777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1031b65-e30e-450c-bd57-4d3a36992072 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:09.048922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1031b65-e30e-450c-bd57-4d3a36992072 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:09.049964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1031b65-e30e-450c-bd57-4d3a36992072 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:09.050659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1031b65-e30e-450c-bd57-4d3a36992072 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:09.233805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-240ead17-4932-41bb-8c3a-29f8fd00fef6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:09.234505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-240ead17-4932-41bb-8c3a-29f8fd00fef6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:09.247392 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-240ead17-4932-41bb-8c3a-29f8fd00fef6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:09.254389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-240ead17-4932-41bb-8c3a-29f8fd00fef6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:09.255560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-240ead17-4932-41bb-8c3a-29f8fd00fef6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:09.256679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-240ead17-4932-41bb-8c3a-29f8fd00fef6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:09.540504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-240ead17-4932-41bb-8c3a-29f8fd00fef6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:09.574095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-240ead17-4932-41bb-8c3a-29f8fd00fef6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.345299 May 25 16:19:09.575564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 116/254] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:09 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:09.581484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1483c28c-8543-4f1a-808b-b232e47f0fbe tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:09.583087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1483c28c-8543-4f1a-808b-b232e47f0fbe tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:09.595658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1483c28c-8543-4f1a-808b-b232e47f0fbe tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:09.596568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1483c28c-8543-4f1a-808b-b232e47f0fbe tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:09.597512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1483c28c-8543-4f1a-808b-b232e47f0fbe tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:09.598094 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1483c28c-8543-4f1a-808b-b232e47f0fbe tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:09.693045 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1483c28c-8543-4f1a-808b-b232e47f0fbe tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:09.698975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a1031b65-e30e-450c-bd57-4d3a36992072 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.668472 May 25 16:19:09.704249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1483c28c-8543-4f1a-808b-b232e47f0fbe tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1136 microversion: 2.1 time: 0.126931 May 25 16:19:09.705927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 139/256] 10.222.0.45 () {60 vars in 1322 bytes} [Wed May 25 16:19:09 2022] POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/os-volume_attachments => generated 194 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:09.706098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 117/256] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:09 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1136 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:09.714566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:09.715710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] [instance: 79123bd5-06c0-402b-86fe-3068c15d1a8d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:09.728799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:09.729935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:09.731195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:09.731914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:09.769091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] HTTP exception thrown: Instance 79123bd5-06c0-402b-86fe-3068c15d1a8d could not be found. May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Request method failure captured: May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d HTTP/1.1 May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlbI6til8OPEw8FGaflpQOyfgrZkd1x_7ZV4wFuridfr4B4Imdu6ldivOEa9T8DTXxRsqJGnEeU3c_CozdhfBut3Uab7rysYMBdf2Q30Oh_Dd1Mx4Kub-xYz5Pk8crkszn19XCiAwY80DGeEyL_e8QMHCOWa8PoSr3PPRzSe9YN6IOM May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 2aca09e1a3bc4926ba6c14f975728e39 May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-VolumesActionsTest-1820165728 May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:19:09.804653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_2aca09e1a3bc4926ba6c14f975728e39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/2aca09e1a3bc4926ba6c14f975728e39"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/2aca09e1a3bc4926ba6c14f975728e39"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/2aca09e1a3bc4926ba6c14f975728e39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-VolumesActionsTest-1820165728 May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 2aca09e1a3bc4926ba6c14f975728e39 May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-VolumesActionsTest-1820165728 May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-VolumesActionsTest-1820165728-project May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 63c237a15880463d9b3850dac6413ceb May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-VolumesActionsTest-1820165728-project May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 79123bd5-06c0-402b-86fe-3068c15d1a8d could not be found. May 25 16:19:09.806176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '79123bd5-06c0-402b-86fe-3068c15d1a8d'} May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 79123bd5-06c0-402b-86fe-3068c15d1a8d could not be found. May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:19:09.807044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 79123bd5-06c0-402b-86fe-3068c15d1a8d could not be found. May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 79123bd5-06c0-402b-86fe-3068c15d1a8d could not be found. May 25 16:19:09.809137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:19:09.811034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] Returning 404 to user: Instance 79123bd5-06c0-402b-86fe-3068c15d1a8d could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:19:09.811034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f6d4d634-453e-4555-a459-d5195a2aea8b tempest-VolumesActionsTest-1820165728 tempest-VolumesActionsTest-1820165728-project] 10.222.0.45 "GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d" status: 404 len: 111 microversion: 2.1 time: 0.099067 May 25 16:19:09.811034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 118/257] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:09 2022] GET /compute/v2.1/servers/79123bd5-06c0-402b-86fe-3068c15d1a8d => generated 111 bytes in 104 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:19:10.594163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2f9b0264-94cd-44db-8542-ff533f2b90c5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:10.595402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2f9b0264-94cd-44db-8542-ff533f2b90c5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:10.611947 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2f9b0264-94cd-44db-8542-ff533f2b90c5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:10.617376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2f9b0264-94cd-44db-8542-ff533f2b90c5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:10.622116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2f9b0264-94cd-44db-8542-ff533f2b90c5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:10.622758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2f9b0264-94cd-44db-8542-ff533f2b90c5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:10.725034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8816911c-d8f5-4d7e-8c31-32b545c52c76 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:10.725660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8816911c-d8f5-4d7e-8c31-32b545c52c76 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:10.735224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8816911c-d8f5-4d7e-8c31-32b545c52c76 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:10.737062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8816911c-d8f5-4d7e-8c31-32b545c52c76 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:10.738990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8816911c-d8f5-4d7e-8c31-32b545c52c76 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:10.739622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8816911c-d8f5-4d7e-8c31-32b545c52c76 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:10.963237 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8816911c-d8f5-4d7e-8c31-32b545c52c76 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:10.976304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8816911c-d8f5-4d7e-8c31-32b545c52c76 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1192 microversion: 2.1 time: 0.255639 May 25 16:19:10.992489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 119/258] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:10 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1192 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:11.032930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2f9b0264-94cd-44db-8542-ff533f2b90c5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:11.050750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2f9b0264-94cd-44db-8542-ff533f2b90c5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.461763 May 25 16:19:11.052243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 140/259] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:10 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:12.010930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c0cc1f9b-4153-4fc8-a7c2-6abcdc99d9be tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:12.011660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c0cc1f9b-4153-4fc8-a7c2-6abcdc99d9be tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:12.020946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c0cc1f9b-4153-4fc8-a7c2-6abcdc99d9be tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:12.021924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c0cc1f9b-4153-4fc8-a7c2-6abcdc99d9be tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:12.023873 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c0cc1f9b-4153-4fc8-a7c2-6abcdc99d9be tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:12.024471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c0cc1f9b-4153-4fc8-a7c2-6abcdc99d9be tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:12.058941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:12.080319 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7be64603-f980-4629-ab5b-28c0aeb01000 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:12.080847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7be64603-f980-4629-ab5b-28c0aeb01000 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:12.094198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7be64603-f980-4629-ab5b-28c0aeb01000 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:12.096296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7be64603-f980-4629-ab5b-28c0aeb01000 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:12.097941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7be64603-f980-4629-ab5b-28c0aeb01000 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:12.098611 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7be64603-f980-4629-ab5b-28c0aeb01000 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:12.132693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c0cc1f9b-4153-4fc8-a7c2-6abcdc99d9be tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:12.144668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c0cc1f9b-4153-4fc8-a7c2-6abcdc99d9be tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1210 microversion: 2.1 time: 0.139995 May 25 16:19:12.145265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 120/260] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:12 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1210 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:12.344816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7be64603-f980-4629-ab5b-28c0aeb01000 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:12.353735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7be64603-f980-4629-ab5b-28c0aeb01000 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.281349 May 25 16:19:12.367374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 141/261] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:12 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:13.162680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9f9f9df9-d769-4b66-bdea-3b58d0bd03bd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:13.163396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9f9f9df9-d769-4b66-bdea-3b58d0bd03bd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:13.170366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f9f9df9-d769-4b66-bdea-3b58d0bd03bd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:13.171057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f9f9df9-d769-4b66-bdea-3b58d0bd03bd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:13.172622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f9f9df9-d769-4b66-bdea-3b58d0bd03bd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:13.173838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f9f9df9-d769-4b66-bdea-3b58d0bd03bd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:13.343938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9f9f9df9-d769-4b66-bdea-3b58d0bd03bd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:13.352422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9f9f9df9-d769-4b66-bdea-3b58d0bd03bd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1277 microversion: 2.1 time: 0.194144 May 25 16:19:13.353012 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 121/262] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:13 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1277 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:13.373985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8bece0be-610d-4391-a57b-3cd095c84048 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:13.375393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8bece0be-610d-4391-a57b-3cd095c84048 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:13.382714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bece0be-610d-4391-a57b-3cd095c84048 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:13.383478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bece0be-610d-4391-a57b-3cd095c84048 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:13.384535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bece0be-610d-4391-a57b-3cd095c84048 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:13.385162 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bece0be-610d-4391-a57b-3cd095c84048 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:13.534146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8bece0be-610d-4391-a57b-3cd095c84048 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:13.543015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8bece0be-610d-4391-a57b-3cd095c84048 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.173010 May 25 16:19:13.553432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 142/263] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:13 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:14.369749 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-723e4eae-066d-4c80-8d99-38e9247b7008 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:14.371389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-723e4eae-066d-4c80-8d99-38e9247b7008 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:14.405102 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-723e4eae-066d-4c80-8d99-38e9247b7008 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:14.405815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-723e4eae-066d-4c80-8d99-38e9247b7008 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:14.411715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-723e4eae-066d-4c80-8d99-38e9247b7008 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:14.412338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-723e4eae-066d-4c80-8d99-38e9247b7008 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:14.560358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6e99c058-72fc-42b5-9c28-59b6593ca2c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:14.560957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6e99c058-72fc-42b5-9c28-59b6593ca2c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:14.564041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=59,nova_api:UPDATE=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:14.569920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e99c058-72fc-42b5-9c28-59b6593ca2c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:14.570613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e99c058-72fc-42b5-9c28-59b6593ca2c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:14.571715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e99c058-72fc-42b5-9c28-59b6593ca2c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:14.572297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e99c058-72fc-42b5-9c28-59b6593ca2c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:14.798018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-723e4eae-066d-4c80-8d99-38e9247b7008 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:14.811397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-723e4eae-066d-4c80-8d99-38e9247b7008 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1277 microversion: 2.1 time: 0.443935 May 25 16:19:14.813647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 122/264] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:14 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1277 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:14.849514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6e99c058-72fc-42b5-9c28-59b6593ca2c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:14.863518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6e99c058-72fc-42b5-9c28-59b6593ca2c8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.305906 May 25 16:19:14.866084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 143/265] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:14 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:14.896385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-2e4ddc01-ef0d-4846-9b2e-92823a36e971 req-caa33c5d-1590-48d2-aafe-f70af6fad57a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b3b920a7-fa28-4150-b889-d9a2ea63bc16", "tag": "533fafe5-1581-4d83-887b-681b728ba38c"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:14.923090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-2e4ddc01-ef0d-4846-9b2e-92823a36e971 req-caa33c5d-1590-48d2-aafe-f70af6fad57a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:14.923848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-2e4ddc01-ef0d-4846-9b2e-92823a36e971 req-caa33c5d-1590-48d2-aafe-f70af6fad57a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:14.944739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-93622633-533c-4a15-a8b5-386389c25d26 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:14.945617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-93622633-533c-4a15-a8b5-386389c25d26 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:14.963322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-93622633-533c-4a15-a8b5-386389c25d26 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:14.966194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-2e4ddc01-ef0d-4846-9b2e-92823a36e971 req-caa33c5d-1590-48d2-aafe-f70af6fad57a service nova] Creating event network-changed:533fafe5-1581-4d83-887b-681b728ba38c for instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 on hv-cinder-84327 May 25 16:19:14.966764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-93622633-533c-4a15-a8b5-386389c25d26 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:14.968203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-93622633-533c-4a15-a8b5-386389c25d26 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:14.968860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-93622633-533c-4a15-a8b5-386389c25d26 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:14.990477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-2e4ddc01-ef0d-4846-9b2e-92823a36e971 req-caa33c5d-1590-48d2-aafe-f70af6fad57a service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098494 May 25 16:19:15.006506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 123/266] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:15.014121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:15.032298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-93622633-533c-4a15-a8b5-386389c25d26 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/ips" status: 200 len: 150 microversion: 2.1 time: 0.093545 May 25 16:19:15.041822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 144/267] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:19:14 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/ips => generated 150 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:15.830652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-da2a420d-3919-4383-9ec1-c02b7fbe3faa tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:15.831569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-da2a420d-3919-4383-9ec1-c02b7fbe3faa tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:15.849781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da2a420d-3919-4383-9ec1-c02b7fbe3faa tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:15.850257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da2a420d-3919-4383-9ec1-c02b7fbe3faa tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:15.851166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da2a420d-3919-4383-9ec1-c02b7fbe3faa tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:15.851665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da2a420d-3919-4383-9ec1-c02b7fbe3faa tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:15.885367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a58a4e2a-6508-49c9-899d-83420c3469f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:15.886739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a58a4e2a-6508-49c9-899d-83420c3469f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:15.895968 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a58a4e2a-6508-49c9-899d-83420c3469f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:15.896776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a58a4e2a-6508-49c9-899d-83420c3469f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:15.897922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a58a4e2a-6508-49c9-899d-83420c3469f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:15.898577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a58a4e2a-6508-49c9-899d-83420c3469f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:16.631562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-da2a420d-3919-4383-9ec1-c02b7fbe3faa tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:16.637927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a58a4e2a-6508-49c9-899d-83420c3469f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:16.647996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-da2a420d-3919-4383-9ec1-c02b7fbe3faa tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1277 microversion: 2.1 time: 0.815412 May 25 16:19:16.650020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 124/268] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:15 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1277 bytes in 824 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:16.671993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a58a4e2a-6508-49c9-899d-83420c3469f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.791146 May 25 16:19:16.673614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 145/269] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:15 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 796 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:17.231049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-acf2e1fa-13ee-45c0-a613-85c526aaf2e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3946a5c7-9a7d-4de6-9801-eb9817e8dd65"}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:17.232307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-acf2e1fa-13ee-45c0-a613-85c526aaf2e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:17.245559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acf2e1fa-13ee-45c0-a613-85c526aaf2e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:17.251270 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acf2e1fa-13ee-45c0-a613-85c526aaf2e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:17.253795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acf2e1fa-13ee-45c0-a613-85c526aaf2e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:17.258972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acf2e1fa-13ee-45c0-a613-85c526aaf2e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:17.667524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7023d5db-01c6-4109-89b0-e25bd04b7bfc tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:17.669416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7023d5db-01c6-4109-89b0-e25bd04b7bfc tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:17.684712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7023d5db-01c6-4109-89b0-e25bd04b7bfc tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:17.690641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7023d5db-01c6-4109-89b0-e25bd04b7bfc tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:17.700995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7023d5db-01c6-4109-89b0-e25bd04b7bfc tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:17.700995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7023d5db-01c6-4109-89b0-e25bd04b7bfc tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:17.819650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:18.089329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-acf2e1fa-13ee-45c0-a613-85c526aaf2e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.862519 May 25 16:19:18.090380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 125/270] 10.222.0.45 () {60 vars in 1322 bytes} [Wed May 25 16:19:17 2022] POST /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 194 bytes in 865 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:18.103277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-69531fff-ba20-46e5-bc69-47e0e68ae1af tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:18.104441 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-69531fff-ba20-46e5-bc69-47e0e68ae1af tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:18.111577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69531fff-ba20-46e5-bc69-47e0e68ae1af tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:18.112378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69531fff-ba20-46e5-bc69-47e0e68ae1af tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:18.113306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69531fff-ba20-46e5-bc69-47e0e68ae1af tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:18.113885 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69531fff-ba20-46e5-bc69-47e0e68ae1af tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:18.138113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7023d5db-01c6-4109-89b0-e25bd04b7bfc tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:18.150515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7023d5db-01c6-4109-89b0-e25bd04b7bfc tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.488565 May 25 16:19:18.153473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 146/271] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:17 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:18.167407 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8aa8c6ae-2f9d-40eb-b4f1-f77f4373d7ac tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:18.188071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8aa8c6ae-2f9d-40eb-b4f1-f77f4373d7ac tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:18.188878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8aa8c6ae-2f9d-40eb-b4f1-f77f4373d7ac tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:18.311140 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8aa8c6ae-2f9d-40eb-b4f1-f77f4373d7ac tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Going to try to stop instance {{(pid=117794) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 25 16:19:18.381604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8aa8c6ae-2f9d-40eb-b4f1-f77f4373d7ac tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/action" status: 202 len: 0 microversion: 2.1 time: 0.223219 May 25 16:19:18.386278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 147/272] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:19:18 2022] POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/action => generated 0 bytes in 229 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:19:18.409526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-10fba989-5d31-481a-8eb1-2ff4e363477b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Action: 'create', calling method: .version_select at 0x7f4a30197670>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-151156222", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNV4svlggSvQkUMEnZJHoK5cS3lnSYV0LUdcmfPqKaPoHdxmU8BDTp+KTgeFu7DblSF/mtWQAQfj4kLbSwo7p3vBljIMWmtlQPGGL4tt9XzCmBSGxm99lbXcxSvOvlwgrQ=="}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:18.417312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-69531fff-ba20-46e5-bc69-47e0e68ae1af tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:18.425045 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-69531fff-ba20-46e5-bc69-47e0e68ae1af tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.327171 May 25 16:19:18.434684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 126/273] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:18 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:18.439386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a5afc053-cff1-40b3-bef6-ee139489833e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:18.440097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a5afc053-cff1-40b3-bef6-ee139489833e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:18.443564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-10fba989-5d31-481a-8eb1-2ff4e363477b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Getting quotas for project 711b13e4ea3e4269a0082811f6708d87. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:19:18.447199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5afc053-cff1-40b3-bef6-ee139489833e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:18.447861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5afc053-cff1-40b3-bef6-ee139489833e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:18.449081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5afc053-cff1-40b3-bef6-ee139489833e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:18.449637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-10fba989-5d31-481a-8eb1-2ff4e363477b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Getting quotas for user e40faab6a34f4d368462cdcafb7bcfea and project 711b13e4ea3e4269a0082811f6708d87. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:19:18.450143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5afc053-cff1-40b3-bef6-ee139489833e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:18.488137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=8 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:18.490087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-10fba989-5d31-481a-8eb1-2ff4e363477b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.100304 May 25 16:19:18.492414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 148/274] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:19:18 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:18.625090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a5afc053-cff1-40b3-bef6-ee139489833e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:18.634955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a5afc053-cff1-40b3-bef6-ee139489833e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.206558 May 25 16:19:18.636398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 127/275] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:18 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:18.653223 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-292389560", "imageRef": "662869ed-1e97-4cc8-b199-bd13ebb02221", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-151156222", "networks": [{"uuid": "37696238-2d32-46c4-9297-e6aa4ae6859f"}]}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:18.694316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4aab05d6-c5e3-438d-8aba-a24485a37377 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:18.695044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4aab05d6-c5e3-438d-8aba-a24485a37377 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:18.703326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4aab05d6-c5e3-438d-8aba-a24485a37377 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:18.703779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4aab05d6-c5e3-438d-8aba-a24485a37377 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:18.707924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4aab05d6-c5e3-438d-8aba-a24485a37377 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:18.709266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4aab05d6-c5e3-438d-8aba-a24485a37377 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:18.709806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.network.neutron [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] validate_networks() for [('37696238-2d32-46c4-9297-e6aa4ae6859f', None, None, None, None, None)] {{(pid=117794) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:19:18.749367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4aab05d6-c5e3-438d-8aba-a24485a37377 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/ips" status: 200 len: 152 microversion: 2.20 time: 0.060261 May 25 16:19:18.756688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 128/276] 10.222.0.45 () {60 vars in 1288 bytes} [Wed May 25 16:19:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/ips => generated 152 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:19:19.174162 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-537fabdd-6068-45ba-a8f6-e93d1925c491 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:19.175037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-537fabdd-6068-45ba-a8f6-e93d1925c491 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:19.190721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-537fabdd-6068-45ba-a8f6-e93d1925c491 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:19.196322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-537fabdd-6068-45ba-a8f6-e93d1925c491 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:19.197775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-537fabdd-6068-45ba-a8f6-e93d1925c491 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:19.198498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-537fabdd-6068-45ba-a8f6-e93d1925c491 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:19.787628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Flavor limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:19:19.787628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Image limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:19:19.792960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Flavor pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:19:19.793534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Image pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:19:19.794006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:19:19.815993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:19.816608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:19.817518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:19.817980 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:19.825924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-537fabdd-6068-45ba-a8f6-e93d1925c491 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:19.838725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-537fabdd-6068-45ba-a8f6-e93d1925c491 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.664858 May 25 16:19:19.839780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 129/277] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:19 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:19.846269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-44700330-5065-46be-a7d4-cff1f08f7e89 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:19.852397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-44700330-5065-46be-a7d4-cff1f08f7e89 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:19.859519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Getting quotas for project 711b13e4ea3e4269a0082811f6708d87. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:19:19.861473 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44700330-5065-46be-a7d4-cff1f08f7e89 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:19.862498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44700330-5065-46be-a7d4-cff1f08f7e89 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:19.863996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44700330-5065-46be-a7d4-cff1f08f7e89 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:19.864711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Getting quotas for user e40faab6a34f4d368462cdcafb7bcfea and project 711b13e4ea3e4269a0082811f6708d87. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:19:19.865025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44700330-5065-46be-a7d4-cff1f08f7e89 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:19.886107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Going to run 1 instances... {{(pid=117794) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:19:19.889823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='662869ed-1e97-4cc8-b199-bd13ebb02221',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117794) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:19:19.962455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f89a27db-13ba-42f2-8f1e-fafdc72805ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.312243 May 25 16:19:19.964101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 149/278] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:19:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 1315 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:19:19.969269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9444915d-8308-4040-b5ba-54f3196f3e5c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:19.970086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9444915d-8308-4040-b5ba-54f3196f3e5c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:19.983692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9444915d-8308-4040-b5ba-54f3196f3e5c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:19.984425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9444915d-8308-4040-b5ba-54f3196f3e5c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:19.985351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9444915d-8308-4040-b5ba-54f3196f3e5c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:19.985948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9444915d-8308-4040-b5ba-54f3196f3e5c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:20.202336 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-44700330-5065-46be-a7d4-cff1f08f7e89 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:20.212883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-44700330-5065-46be-a7d4-cff1f08f7e89 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.370127 May 25 16:19:20.236364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 130/279] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:19 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:20.244272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dc1461a1-1a60-45c3-8080-6fe6f126cfce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:20.246986 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dc1461a1-1a60-45c3-8080-6fe6f126cfce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:20.250803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9444915d-8308-4040-b5ba-54f3196f3e5c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:20.261073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9444915d-8308-4040-b5ba-54f3196f3e5c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.295817 May 25 16:19:20.262390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 150/280] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:19 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:20.387482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dc1461a1-1a60-45c3-8080-6fe6f126cfce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:20.402886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dc1461a1-1a60-45c3-8080-6fe6f126cfce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1257 microversion: 2.1 time: 0.165190 May 25 16:19:20.403518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 131/281] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:20 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1257 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:20.856589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-45347db8-90b0-47df-a308-e019833b72ab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:20.857279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-45347db8-90b0-47df-a308-e019833b72ab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:20.865693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-45347db8-90b0-47df-a308-e019833b72ab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:20.866383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-45347db8-90b0-47df-a308-e019833b72ab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:20.867168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-45347db8-90b0-47df-a308-e019833b72ab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:20.868155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-45347db8-90b0-47df-a308-e019833b72ab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:21.057333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-45347db8-90b0-47df-a308-e019833b72ab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:21.066760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-45347db8-90b0-47df-a308-e019833b72ab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.214560 May 25 16:19:21.071259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 151/282] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:20 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:21.234403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5b067d55-d2f6-4be1-a586-9cbe3e94db82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:21.235180 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5b067d55-d2f6-4be1-a586-9cbe3e94db82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:21.244800 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b067d55-d2f6-4be1-a586-9cbe3e94db82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:21.245430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b067d55-d2f6-4be1-a586-9cbe3e94db82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:21.246218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b067d55-d2f6-4be1-a586-9cbe3e94db82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:21.246672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b067d55-d2f6-4be1-a586-9cbe3e94db82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:21.279432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a1f639ff-35bb-43ec-ad47-feefbab132aa tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:21.280101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a1f639ff-35bb-43ec-ad47-feefbab132aa tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:21.294443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1f639ff-35bb-43ec-ad47-feefbab132aa tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:21.294926 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1f639ff-35bb-43ec-ad47-feefbab132aa tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:21.295735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1f639ff-35bb-43ec-ad47-feefbab132aa tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:21.296171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1f639ff-35bb-43ec-ad47-feefbab132aa tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:21.622965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a1f639ff-35bb-43ec-ad47-feefbab132aa tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:21.636741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a1f639ff-35bb-43ec-ad47-feefbab132aa tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.361191 May 25 16:19:21.639652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 152/283] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:21 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:21.651479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-58f2aa61-37e2-495c-aacd-113271797450 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:21.652569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-58f2aa61-37e2-495c-aacd-113271797450 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:21.660748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58f2aa61-37e2-495c-aacd-113271797450 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:21.661654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58f2aa61-37e2-495c-aacd-113271797450 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:21.662634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58f2aa61-37e2-495c-aacd-113271797450 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:21.663428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58f2aa61-37e2-495c-aacd-113271797450 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:21.673635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5b067d55-d2f6-4be1-a586-9cbe3e94db82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:21.692497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-58f2aa61-37e2-495c-aacd-113271797450 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lazy-loading 'flavor' on Instance uuid 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:19:21.699197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5b067d55-d2f6-4be1-a586-9cbe3e94db82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.473023 May 25 16:19:21.699962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 132/284] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:21 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:21.736301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0821c129-5bcf-497f-8dd1-0caa5c27151f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:21.739738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0821c129-5bcf-497f-8dd1-0caa5c27151f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:21.760455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0821c129-5bcf-497f-8dd1-0caa5c27151f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:21.761267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0821c129-5bcf-497f-8dd1-0caa5c27151f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:21.778085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0821c129-5bcf-497f-8dd1-0caa5c27151f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:21.778946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0821c129-5bcf-497f-8dd1-0caa5c27151f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:21.877585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0821c129-5bcf-497f-8dd1-0caa5c27151f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:21.890858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0821c129-5bcf-497f-8dd1-0caa5c27151f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1309 microversion: 2.1 time: 0.189626 May 25 16:19:21.899707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 133/285] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:21 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1309 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:22.058001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-58f2aa61-37e2-495c-aacd-113271797450 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "POST /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/action" status: 202 len: 0 microversion: 2.20 time: 0.414843 May 25 16:19:22.058869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 153/286] 10.222.0.45 () {62 vars in 1318 bytes} [Wed May 25 16:19:21 2022] POST /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/action => generated 0 bytes in 418 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 25 16:19:22.068052 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:22.069158 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:22.076538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:22.077530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:22.078518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:22.079361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:22.097458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-91743ce9-1abe-404b-b49a-2ccc5fe47ab0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:22.098533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-91743ce9-1abe-404b-b49a-2ccc5fe47ab0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:22.121623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-91743ce9-1abe-404b-b49a-2ccc5fe47ab0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:22.122124 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-91743ce9-1abe-404b-b49a-2ccc5fe47ab0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:22.123044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-91743ce9-1abe-404b-b49a-2ccc5fe47ab0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:22.123497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-91743ce9-1abe-404b-b49a-2ccc5fe47ab0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:22.392490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:22.407736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:22.409495 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:22.411369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d86b6901-136e-474d-ab65-081e3d58668c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.347920 May 25 16:19:22.412265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 134/287] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:22 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:22.416652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-91743ce9-1abe-404b-b49a-2ccc5fe47ab0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:22.429385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-91743ce9-1abe-404b-b49a-2ccc5fe47ab0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.343730 May 25 16:19:22.430298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 154/288] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:22 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:22.658682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d4ef3892-9f39-47aa-93c0-d5301fcbd8dd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:22.659261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d4ef3892-9f39-47aa-93c0-d5301fcbd8dd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:22.666353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4ef3892-9f39-47aa-93c0-d5301fcbd8dd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:22.667072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4ef3892-9f39-47aa-93c0-d5301fcbd8dd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:22.667882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4ef3892-9f39-47aa-93c0-d5301fcbd8dd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:22.668315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4ef3892-9f39-47aa-93c0-d5301fcbd8dd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:22.715973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1fc31417-2bf0-4096-aacc-c2fade68408e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:22.716763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1fc31417-2bf0-4096-aacc-c2fade68408e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:22.729312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fc31417-2bf0-4096-aacc-c2fade68408e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:22.729904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fc31417-2bf0-4096-aacc-c2fade68408e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:22.730747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fc31417-2bf0-4096-aacc-c2fade68408e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:22.732011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fc31417-2bf0-4096-aacc-c2fade68408e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.053404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d4ef3892-9f39-47aa-93c0-d5301fcbd8dd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:23.067283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d4ef3892-9f39-47aa-93c0-d5301fcbd8dd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.415077 May 25 16:19:23.080714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 135/289] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:22 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:23.095949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a59be809-6659-499c-b88f-fa98fa227390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:23.100247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a59be809-6659-499c-b88f-fa98fa227390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:23.129765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a59be809-6659-499c-b88f-fa98fa227390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.134772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a59be809-6659-499c-b88f-fa98fa227390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.136847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a59be809-6659-499c-b88f-fa98fa227390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.139683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a59be809-6659-499c-b88f-fa98fa227390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.154355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1fc31417-2bf0-4096-aacc-c2fade68408e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:23.164893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1fc31417-2bf0-4096-aacc-c2fade68408e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1500 microversion: 2.1 time: 0.450889 May 25 16:19:23.179320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 155/290] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:22 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1500 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:23.194992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ea20f0f2-d8f8-4ff5-bc3a-4c5c04efc8ca"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:23.198802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:23.205270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.205868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.206888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.207110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.234629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'attach_volume' instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d while it is in vm_state rescued May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: POST /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d/os-volume_attachments HTTP/1.1 May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 74 May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:19:23.236477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {"volumeAttachment": {"volumeId": "ea20f0f2-d8f8-4ff5-bc3a-4c5c04efc8ca"}} May 25 16:19:23.238399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Cannot 'attach_volume' instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d while it is in vm_state rescued May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': '69fc8c5b-82f0-4a03-8dab-381fdee5884d', 'body': {'volumeAttachment': {'volumeId': 'ea20f0f2-d8f8-4ff5-bc3a-4c5c04efc8ca'}}} May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d in vm_state rescued. Cannot attach_volume while the instance is in this state. May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:19:23.239374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d while it is in vm_state rescued May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:19:23.241082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:19:23.243311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:19:23.243311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:19:23.243311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d while it is in vm_state rescued May 25 16:19:23.243311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:19:23.243311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'attach_volume' instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d while it is in vm_state rescued {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:19:23.243311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7a4d3c7a-f71d-4f47-a5c9-6ab747ad1983 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.055759 May 25 16:19:23.243311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 156/291] 10.222.0.45 () {60 vars in 1322 bytes} [Wed May 25 16:19:23 2022] POST /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d/os-volume_attachments => generated 152 bytes in 57 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:19:23.248235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3ef99121-6de9-486b-a4ec-f6d26628e664 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:23.248864 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3ef99121-6de9-486b-a4ec-f6d26628e664 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:23.255892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3ef99121-6de9-486b-a4ec-f6d26628e664 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.256344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3ef99121-6de9-486b-a4ec-f6d26628e664 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.257123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3ef99121-6de9-486b-a4ec-f6d26628e664 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.257558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3ef99121-6de9-486b-a4ec-f6d26628e664 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.258312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a59be809-6659-499c-b88f-fa98fa227390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:23.269822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a59be809-6659-499c-b88f-fa98fa227390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1315 microversion: 2.1 time: 0.187544 May 25 16:19:23.280188 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 136/292] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:23 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1315 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:23.337997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3ef99121-6de9-486b-a4ec-f6d26628e664 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d/action" status: 202 len: 0 microversion: 2.1 time: 0.093485 May 25 16:19:23.341728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 157/293] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:19:23 2022] POST /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d/action => generated 0 bytes in 95 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 25 16:19:23.349784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fc2085f6-2f66-41f1-9028-21e4e416fb03 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:23.350410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fc2085f6-2f66-41f1-9028-21e4e416fb03 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:23.358509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fc2085f6-2f66-41f1-9028-21e4e416fb03 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.359305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fc2085f6-2f66-41f1-9028-21e4e416fb03 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.360625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fc2085f6-2f66-41f1-9028-21e4e416fb03 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.361181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fc2085f6-2f66-41f1-9028-21e4e416fb03 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.445068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:23.445068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:23.456458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.466143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.467269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.467992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.586390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fc2085f6-2f66-41f1-9028-21e4e416fb03 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:23.588668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:23.603833 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fc2085f6-2f66-41f1-9028-21e4e416fb03 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1508 microversion: 2.1 time: 0.259647 May 25 16:19:23.604827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 137/294] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:23 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1508 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:23.613417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bd249d42-00bc-45e8-89a9-5b8d1424d16b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:23.614308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bd249d42-00bc-45e8-89a9-5b8d1424d16b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:23.625130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd249d42-00bc-45e8-89a9-5b8d1424d16b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.625903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd249d42-00bc-45e8-89a9-5b8d1424d16b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.626763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd249d42-00bc-45e8-89a9-5b8d1424d16b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:23.627268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd249d42-00bc-45e8-89a9-5b8d1424d16b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:23.687459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:23.702756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:23.711814 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:23.712951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-982844ed-3174-4b07-a863-c294d7eadb63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.286134 May 25 16:19:23.713506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 158/295] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:23 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:23.788170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bd249d42-00bc-45e8-89a9-5b8d1424d16b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:23.797265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bd249d42-00bc-45e8-89a9-5b8d1424d16b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.190577 May 25 16:19:23.798579 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 138/296] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:23 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:24.085595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-37dc1c4b-0543-4ece-b3ab-283edf37fc50 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:24.086522 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-37dc1c4b-0543-4ece-b3ab-283edf37fc50 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:24.093538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-37dc1c4b-0543-4ece-b3ab-283edf37fc50 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:24.094221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-37dc1c4b-0543-4ece-b3ab-283edf37fc50 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:24.095194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-37dc1c4b-0543-4ece-b3ab-283edf37fc50 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:24.095806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-37dc1c4b-0543-4ece-b3ab-283edf37fc50 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:24.113668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:24.263908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-37dc1c4b-0543-4ece-b3ab-283edf37fc50 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:24.272367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-37dc1c4b-0543-4ece-b3ab-283edf37fc50 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.190905 May 25 16:19:24.283544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 159/297] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:24 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:24.289961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4845fcbe-28a8-4db4-b7e5-3cc1a614e245 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:24.290815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4845fcbe-28a8-4db4-b7e5-3cc1a614e245 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:24.320442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4845fcbe-28a8-4db4-b7e5-3cc1a614e245 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:24.322625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4845fcbe-28a8-4db4-b7e5-3cc1a614e245 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:24.327006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4845fcbe-28a8-4db4-b7e5-3cc1a614e245 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:24.328847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4845fcbe-28a8-4db4-b7e5-3cc1a614e245 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:24.439376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4845fcbe-28a8-4db4-b7e5-3cc1a614e245 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:24.459302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4845fcbe-28a8-4db4-b7e5-3cc1a614e245 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1315 microversion: 2.1 time: 0.172003 May 25 16:19:24.469987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 139/298] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:24 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1315 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:24.624372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2392bdf7-5807-414e-9a7a-c9bae8cfde90 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:24.625060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2392bdf7-5807-414e-9a7a-c9bae8cfde90 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:24.632706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2392bdf7-5807-414e-9a7a-c9bae8cfde90 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:24.633671 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2392bdf7-5807-414e-9a7a-c9bae8cfde90 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:24.634669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2392bdf7-5807-414e-9a7a-c9bae8cfde90 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:24.635346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2392bdf7-5807-414e-9a7a-c9bae8cfde90 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:24.734245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:24.735709 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:24.748494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:24.751178 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:24.752990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:24.753429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:25.008199 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2392bdf7-5807-414e-9a7a-c9bae8cfde90 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:25.027335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2392bdf7-5807-414e-9a7a-c9bae8cfde90 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1508 microversion: 2.1 time: 0.404502 May 25 16:19:25.030009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 160/299] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:24 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1508 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:25.037460 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8299581b-56e8-4e3a-bf59-da3f889949b2 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:25.039352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8299581b-56e8-4e3a-bf59-da3f889949b2 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:25.050168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8299581b-56e8-4e3a-bf59-da3f889949b2 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:25.051142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8299581b-56e8-4e3a-bf59-da3f889949b2 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:25.052476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8299581b-56e8-4e3a-bf59-da3f889949b2 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:25.053630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8299581b-56e8-4e3a-bf59-da3f889949b2 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:25.066320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:25.082959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:25.090143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:25.093629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b395d8a8-d788-46e0-b393-edc141d99fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.364124 May 25 16:19:25.107919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 140/300] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:24 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:25.296167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ec13907e-6264-4dea-a04a-df210c984608 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:25.297526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ec13907e-6264-4dea-a04a-df210c984608 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:25.306218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec13907e-6264-4dea-a04a-df210c984608 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:25.306985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec13907e-6264-4dea-a04a-df210c984608 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:25.308269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec13907e-6264-4dea-a04a-df210c984608 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:25.309197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec13907e-6264-4dea-a04a-df210c984608 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:25.507404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8299581b-56e8-4e3a-bf59-da3f889949b2 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:25.525750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8299581b-56e8-4e3a-bf59-da3f889949b2 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.494156 May 25 16:19:25.526343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 161/301] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:25 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:25.541465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dec05174-330f-4764-9365-0097a73c9924 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:25.542070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dec05174-330f-4764-9365-0097a73c9924 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:25.573343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dec05174-330f-4764-9365-0097a73c9924 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:25.573793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dec05174-330f-4764-9365-0097a73c9924 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:25.574573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dec05174-330f-4764-9365-0097a73c9924 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:25.575004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dec05174-330f-4764-9365-0097a73c9924 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:25.728055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ec13907e-6264-4dea-a04a-df210c984608 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:25.749669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ec13907e-6264-4dea-a04a-df210c984608 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.460380 May 25 16:19:25.759323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 141/302] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:25 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:26.048000 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-31896cc9-972f-4778-99f0-b6c6823158dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:26.048908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-31896cc9-972f-4778-99f0-b6c6823158dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:26.054021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dec05174-330f-4764-9365-0097a73c9924 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:26.061420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31896cc9-972f-4778-99f0-b6c6823158dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:26.068289 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dec05174-330f-4764-9365-0097a73c9924 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.530862 May 25 16:19:26.073106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31896cc9-972f-4778-99f0-b6c6823158dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:26.075338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 162/303] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:25 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:26.086281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31896cc9-972f-4778-99f0-b6c6823158dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:26.086799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31896cc9-972f-4778-99f0-b6c6823158dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:26.116491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:26.117990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:26.133519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:26.137992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:26.141315 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:26.142667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:26.733252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:26.746747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:26.753405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:26.755047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f31ad573-1245-4f64-aa80-d81c2a615f75 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.644440 May 25 16:19:26.755910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 163/304] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:26.771721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-33f9e8ed-dd42-48b5-bb4b-2295fa1217e5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: .version_select at 0x7f4a30083c10>, body: {"keypair": {"name": "tempest-keypair-1616215641", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCOelDwsc+gupJ8Cq+xW1CMQuq+U7pqq2bvJTjWxRgyDW6N0Pp6krQycirGSVKBvJpbIkwJUnA+g90+WvleqvBUYOfnEGuMOzLPrT761+FwLREyGjm8bzgyedSRYnq2qNg=="}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:26.838935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-33f9e8ed-dd42-48b5-bb4b-2295fa1217e5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:19:26.856041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-33f9e8ed-dd42-48b5-bb4b-2295fa1217e5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for user a0c9f3f10fd142439521e4b36cf33157 and project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:19:26.990264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-33f9e8ed-dd42-48b5-bb4b-2295fa1217e5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.228034 May 25 16:19:26.997681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 164/305] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:19:26 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:27.008746 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-31896cc9-972f-4778-99f0-b6c6823158dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:27.011600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2e31fdc9-41d2-42a1-bfac-afde6fb9c7bb tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:27.012528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2e31fdc9-41d2-42a1-bfac-afde6fb9c7bb tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:27.035998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e31fdc9-41d2-42a1-bfac-afde6fb9c7bb tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:27.036233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-31896cc9-972f-4778-99f0-b6c6823158dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1508 microversion: 2.1 time: 0.992584 May 25 16:19:27.037294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 142/306] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:26 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1508 bytes in 994 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:27.039370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e31fdc9-41d2-42a1-bfac-afde6fb9c7bb tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:27.042625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e31fdc9-41d2-42a1-bfac-afde6fb9c7bb tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:27.045444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e31fdc9-41d2-42a1-bfac-afde6fb9c7bb tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:27.051342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d0010815-b770-49ec-9f5a-1684f7e0e770 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:27.052303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d0010815-b770-49ec-9f5a-1684f7e0e770 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:27.061192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0010815-b770-49ec-9f5a-1684f7e0e770 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:27.061959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0010815-b770-49ec-9f5a-1684f7e0e770 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:27.073055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0010815-b770-49ec-9f5a-1684f7e0e770 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:27.082293 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0010815-b770-49ec-9f5a-1684f7e0e770 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:28.004525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2e31fdc9-41d2-42a1-bfac-afde6fb9c7bb tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:28.015732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2e31fdc9-41d2-42a1-bfac-afde6fb9c7bb tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 1.016527 May 25 16:19:28.022382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 165/307] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:26 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 1024 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:28.042881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1103e7ed-60d4-44c9-90a0-08dbbb3b2884 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:28.045116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1103e7ed-60d4-44c9-90a0-08dbbb3b2884 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:28.064090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1103e7ed-60d4-44c9-90a0-08dbbb3b2884 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:28.065197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1103e7ed-60d4-44c9-90a0-08dbbb3b2884 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:28.069502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d0010815-b770-49ec-9f5a-1684f7e0e770 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:28.070880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1103e7ed-60d4-44c9-90a0-08dbbb3b2884 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:28.072851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1103e7ed-60d4-44c9-90a0-08dbbb3b2884 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:28.103422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d0010815-b770-49ec-9f5a-1684f7e0e770 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 1.044648 May 25 16:19:28.104694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 143/308] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:27 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 1067 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:28.124172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:28.135673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:28.155750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:28.156203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:28.157076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:28.160066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:29.167961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1103e7ed-60d4-44c9-90a0-08dbbb3b2884 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:29.171386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:29.183064 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1103e7ed-60d4-44c9-90a0-08dbbb3b2884 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 1.157583 May 25 16:19:29.187438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:29.189808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:29.192017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 166/309] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:28 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 1168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:29.193984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-716ecf05-de97-46eb-8b32-35145bacea0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 1.073930 May 25 16:19:29.212343 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 144/310] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 1093 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:29.213800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f4616fe1-8d49-4785-8846-569f6a1b9b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:29.215592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f4616fe1-8d49-4785-8846-569f6a1b9b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:29.218483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-e1a010b5-550c-4ce2-8cca-d30d64e6ca68 req-5ba89352-ee07-4bc4-ba93-ec8b762945a5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b3b920a7-fa28-4150-b889-d9a2ea63bc16", "name": "network-vif-plugged", "status": "completed", "tag": "533fafe5-1581-4d83-887b-681b728ba38c"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:29.235923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f4616fe1-8d49-4785-8846-569f6a1b9b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:29.236757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-e1a010b5-550c-4ce2-8cca-d30d64e6ca68 req-5ba89352-ee07-4bc4-ba93-ec8b762945a5 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:29.237737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f4616fe1-8d49-4785-8846-569f6a1b9b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:29.238560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f4616fe1-8d49-4785-8846-569f6a1b9b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:29.241013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f4616fe1-8d49-4785-8846-569f6a1b9b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:29.241802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-e1a010b5-550c-4ce2-8cca-d30d64e6ca68 req-5ba89352-ee07-4bc4-ba93-ec8b762945a5 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:29.268168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-e1a010b5-550c-4ce2-8cca-d30d64e6ca68 req-5ba89352-ee07-4bc4-ba93-ec8b762945a5 service nova] Creating event network-vif-plugged:533fafe5-1581-4d83-887b-681b728ba38c for instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 on hv-cinder-84327 May 25 16:19:29.290305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-e1a010b5-550c-4ce2-8cca-d30d64e6ca68 req-5ba89352-ee07-4bc4-ba93-ec8b762945a5 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076024 May 25 16:19:29.300176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 145/311] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:29.314057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f43314c8-8a73-4981-a18f-a3a99de2d5c4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:29.325487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f43314c8-8a73-4981-a18f-a3a99de2d5c4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:29.346409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f43314c8-8a73-4981-a18f-a3a99de2d5c4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:29.353905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f43314c8-8a73-4981-a18f-a3a99de2d5c4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:29.356557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f43314c8-8a73-4981-a18f-a3a99de2d5c4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:29.357237 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f43314c8-8a73-4981-a18f-a3a99de2d5c4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:29.763359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f43314c8-8a73-4981-a18f-a3a99de2d5c4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:29.781176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f43314c8-8a73-4981-a18f-a3a99de2d5c4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.473916 May 25 16:19:29.784876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 146/312] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:29 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:29.792023 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1370eed0-8de4-46bf-965b-ecba9c62c1cf tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:29.799129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1370eed0-8de4-46bf-965b-ecba9c62c1cf tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:29.800981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f4616fe1-8d49-4785-8846-569f6a1b9b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:29.811455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1370eed0-8de4-46bf-965b-ecba9c62c1cf tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:29.814294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f4616fe1-8d49-4785-8846-569f6a1b9b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1508 microversion: 2.1 time: 0.617001 May 25 16:19:29.815391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1370eed0-8de4-46bf-965b-ecba9c62c1cf tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:29.816437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1370eed0-8de4-46bf-965b-ecba9c62c1cf tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:29.817087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1370eed0-8de4-46bf-965b-ecba9c62c1cf tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:29.817441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 167/313] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:29 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1508 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:29.823483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-1f438902-dc85-49e8-820b-a7e2eb38ec25 req-efb39d55-2934-4eb1-8ddd-175ef603f999 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d86354dc-f3d4-409d-b331-9f708947be2a", "tag": "e2c0938e-f9fb-4796-87fa-103cbcfd2536"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:29.846066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-1f438902-dc85-49e8-820b-a7e2eb38ec25 req-efb39d55-2934-4eb1-8ddd-175ef603f999 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:29.846730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-1f438902-dc85-49e8-820b-a7e2eb38ec25 req-efb39d55-2934-4eb1-8ddd-175ef603f999 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:29.852873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:29.885960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-1f438902-dc85-49e8-820b-a7e2eb38ec25 req-efb39d55-2934-4eb1-8ddd-175ef603f999 service nova] Creating event network-changed:e2c0938e-f9fb-4796-87fa-103cbcfd2536 for instance d86354dc-f3d4-409d-b331-9f708947be2a on hv-cinder-84327 May 25 16:19:29.901656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-1f438902-dc85-49e8-820b-a7e2eb38ec25 req-efb39d55-2934-4eb1-8ddd-175ef603f999 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082858 May 25 16:19:29.902272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 168/314] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:30.097439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1370eed0-8de4-46bf-965b-ecba9c62c1cf tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:30.107814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1370eed0-8de4-46bf-965b-ecba9c62c1cf tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.321562 May 25 16:19:30.111072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 147/315] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:29 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:30.210490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-87a4c339-1d85-4dbb-bcf7-ef7315efedcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:30.212126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-87a4c339-1d85-4dbb-bcf7-ef7315efedcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:30.215030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:30.215797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:30.221114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87a4c339-1d85-4dbb-bcf7-ef7315efedcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:30.221925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87a4c339-1d85-4dbb-bcf7-ef7315efedcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:30.222948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87a4c339-1d85-4dbb-bcf7-ef7315efedcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:30.223558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87a4c339-1d85-4dbb-bcf7-ef7315efedcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:30.224569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:30.227648 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:30.228538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:30.229088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:30.402730 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:30.702596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-87a4c339-1d85-4dbb-bcf7-ef7315efedcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:30.709940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:30.711308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-87a4c339-1d85-4dbb-bcf7-ef7315efedcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.506204 May 25 16:19:30.721300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:30.722393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 169/316] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:30 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:30.722907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:30.724447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fe8295a1-ef48-4b0c-b580-428f87bc4440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.514562 May 25 16:19:30.735230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 148/317] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:30 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:19:30.738248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-740215357", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "aaf08a79-3442-476f-b613-a02c12c36d8d"}], "security_groups": [{"name": "tempest-securitygroup--214181161"}], "key_name": "tempest-keypair-1616215641", "user_data": "}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:30.801536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bede6438-097e-43db-8078-45e7c2bf02f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:30.802540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bede6438-097e-43db-8078-45e7c2bf02f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:30.813172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bede6438-097e-43db-8078-45e7c2bf02f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:30.814333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bede6438-097e-43db-8078-45e7c2bf02f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:30.815884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bede6438-097e-43db-8078-45e7c2bf02f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:30.817302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bede6438-097e-43db-8078-45e7c2bf02f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:31.058165 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bede6438-097e-43db-8078-45e7c2bf02f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:31.069508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bede6438-097e-43db-8078-45e7c2bf02f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.272116 May 25 16:19:31.080116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 149/318] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:30 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:31.090725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4d9ff7db-7f2c-4ba3-984f-cce9850f44fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:31.091876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4d9ff7db-7f2c-4ba3-984f-cce9850f44fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:31.094136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:31.105854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d9ff7db-7f2c-4ba3-984f-cce9850f44fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:31.106823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d9ff7db-7f2c-4ba3-984f-cce9850f44fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:31.107757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d9ff7db-7f2c-4ba3-984f-cce9850f44fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:31.108405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d9ff7db-7f2c-4ba3-984f-cce9850f44fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:31.371989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4d9ff7db-7f2c-4ba3-984f-cce9850f44fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:31.383292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4d9ff7db-7f2c-4ba3-984f-cce9850f44fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1508 microversion: 2.1 time: 0.301998 May 25 16:19:31.387870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 150/319] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:31 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1508 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:31.392948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d8a6d002-3288-4e02-a8a6-804ac9af3800 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:31.404824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d8a6d002-3288-4e02-a8a6-804ac9af3800 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:31.446922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8a6d002-3288-4e02-a8a6-804ac9af3800 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:31.447814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8a6d002-3288-4e02-a8a6-804ac9af3800 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:31.449724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8a6d002-3288-4e02-a8a6-804ac9af3800 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:31.450757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8a6d002-3288-4e02-a8a6-804ac9af3800 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:31.698116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d8a6d002-3288-4e02-a8a6-804ac9af3800 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:31.708116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d8a6d002-3288-4e02-a8a6-804ac9af3800 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.318830 May 25 16:19:31.709025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 151/320] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:31 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:31.737233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d5d4c702-ed01-4d9c-8516-8f8832b4b82d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:31.739996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d5d4c702-ed01-4d9c-8516-8f8832b4b82d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:31.753571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d5d4c702-ed01-4d9c-8516-8f8832b4b82d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:31.755810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d5d4c702-ed01-4d9c-8516-8f8832b4b82d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:31.755810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d5d4c702-ed01-4d9c-8516-8f8832b4b82d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:31.755810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d5d4c702-ed01-4d9c-8516-8f8832b4b82d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:32.005165 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d5d4c702-ed01-4d9c-8516-8f8832b4b82d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:32.013522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d5d4c702-ed01-4d9c-8516-8f8832b4b82d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.280888 May 25 16:19:32.023956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 152/321] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:31 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:32.031451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:32.032651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:32.048158 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:32.049249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:32.050960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:32.051866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:32.297335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:32.306759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:32.308719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:32.310137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-97c54e04-b070-41a2-b359-f2bd3a40163b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.284699 May 25 16:19:32.321911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 153/322] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:32 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:32.328598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-312ad84b-cbc4-4cf9-a296-dc483ca1e71a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:32.330446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-312ad84b-cbc4-4cf9-a296-dc483ca1e71a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:32.339732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-312ad84b-cbc4-4cf9-a296-dc483ca1e71a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:32.340414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-312ad84b-cbc4-4cf9-a296-dc483ca1e71a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:32.343027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-312ad84b-cbc4-4cf9-a296-dc483ca1e71a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:32.343851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-312ad84b-cbc4-4cf9-a296-dc483ca1e71a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:32.516701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-312ad84b-cbc4-4cf9-a296-dc483ca1e71a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:32.526413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-312ad84b-cbc4-4cf9-a296-dc483ca1e71a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.203072 May 25 16:19:32.544499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 154/323] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:32 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:32.562983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-74472f62-ea7c-497d-a883-ff1c51ae8599 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:32.577931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-74472f62-ea7c-497d-a883-ff1c51ae8599 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:32.589004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74472f62-ea7c-497d-a883-ff1c51ae8599 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:32.589539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74472f62-ea7c-497d-a883-ff1c51ae8599 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:32.590403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74472f62-ea7c-497d-a883-ff1c51ae8599 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:32.590907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74472f62-ea7c-497d-a883-ff1c51ae8599 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:32.594015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:32.889451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-74472f62-ea7c-497d-a883-ff1c51ae8599 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:32.900973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-74472f62-ea7c-497d-a883-ff1c51ae8599 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1514 microversion: 2.1 time: 0.343898 May 25 16:19:32.902223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 155/324] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:32 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1514 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:32.907266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-35b0de66-8363-4ea9-b7ba-994c2825e24d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:32.908248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-35b0de66-8363-4ea9-b7ba-994c2825e24d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:32.932152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-35b0de66-8363-4ea9-b7ba-994c2825e24d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:32.938575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-35b0de66-8363-4ea9-b7ba-994c2825e24d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:32.939545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-35b0de66-8363-4ea9-b7ba-994c2825e24d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:32.940051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-35b0de66-8363-4ea9-b7ba-994c2825e24d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:33.216745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-35b0de66-8363-4ea9-b7ba-994c2825e24d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:33.227712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-35b0de66-8363-4ea9-b7ba-994c2825e24d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.323909 May 25 16:19:33.229697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 156/325] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:32 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:33.234926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ed0cc8d4-b68e-400c-a997-d4d366842853 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:33.235845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ed0cc8d4-b68e-400c-a997-d4d366842853 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:33.242865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed0cc8d4-b68e-400c-a997-d4d366842853 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:33.243778 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed0cc8d4-b68e-400c-a997-d4d366842853 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:33.245044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed0cc8d4-b68e-400c-a997-d4d366842853 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:33.245871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed0cc8d4-b68e-400c-a997-d4d366842853 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:33.472838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ed0cc8d4-b68e-400c-a997-d4d366842853 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:33.483588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ed0cc8d4-b68e-400c-a997-d4d366842853 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.252219 May 25 16:19:33.484468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 157/326] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:33 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:33.495105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:33.495105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:33.497303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:33.497861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:33.498713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:33.499289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:33.757724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:33.770853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:33.772756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:33.774006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-51c4c458-b282-4a7e-b3d2-8e1240ca56a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.288704 May 25 16:19:33.786588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 158/327] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:33 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:33.794998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-2033279828", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "networks": [{"uuid": "76fa603d-f376-4c17-aeff-adef7293c5c4"}]}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:36.884372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=14 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:37.127095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] validate_networks() for [('76fa603d-f376-4c17-aeff-adef7293c5c4', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:19:37.239957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.network.neutron [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] validate_networks() for [('aaf08a79-3442-476f-b613-a02c12c36d8d', None, None, None, None, None)] {{(pid=117794) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:19:37.827254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:19:37.827856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:19:37.828333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:19:37.828736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:19:37.829140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:19:37.838321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:37.838755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:37.839765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:37.840228 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:37.858130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Getting quotas for project 073368e447af465abb41aa556ee46dc9. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:19:37.861789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Flavor limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:19:37.862085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Getting quotas for user b53b45a485b14e0b9273c019dc9ec206 and project 073368e447af465abb41aa556ee46dc9. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:19:37.862698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Image limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:19:37.863712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Flavor pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:19:37.864166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Image pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:19:37.864874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:19:37.874578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:19:37.876802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:37.876937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:19:37.877321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:37.878116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:37.878626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:37.899272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:19:37.903931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for user a0c9f3f10fd142439521e4b36cf33157 and project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:19:37.906649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:37.927026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6175dfa5-a906-4c79-92dc-13fa5796d85f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.138851 May 25 16:19:37.927807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 159/328] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:19:33 2022] POST /compute/v2.1/servers => generated 380 bytes in 4141 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:19:37.932657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7d8206f6-1e8e-40a7-97f7-f981096f88c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:37.933144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Going to run 1 instances... {{(pid=117794) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:19:37.933273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7d8206f6-1e8e-40a7-97f7-f981096f88c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:37.936711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117794) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:19:37.939882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d8206f6-1e8e-40a7-97f7-f981096f88c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:37.940390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d8206f6-1e8e-40a7-97f7-f981096f88c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:37.941186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d8206f6-1e8e-40a7-97f7-f981096f88c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:37.941616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d8206f6-1e8e-40a7-97f7-f981096f88c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:37.998766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-90ed29f3-e415-4fcb-943a-53257d81654f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 7.265244 May 25 16:19:37.999983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 170/329] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:19:30 2022] POST /compute/v2.1/servers => generated 401 bytes in 7277 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:19:38.004972 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ba14bdf4-b91a-4866-a1ff-fa9c9119f0e8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:38.006060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ba14bdf4-b91a-4866-a1ff-fa9c9119f0e8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:38.017219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ba14bdf4-b91a-4866-a1ff-fa9c9119f0e8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.017926 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ba14bdf4-b91a-4866-a1ff-fa9c9119f0e8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.020444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ba14bdf4-b91a-4866-a1ff-fa9c9119f0e8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.021202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ba14bdf4-b91a-4866-a1ff-fa9c9119f0e8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.167404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7d8206f6-1e8e-40a7-97f7-f981096f88c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:38.179248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7d8206f6-1e8e-40a7-97f7-f981096f88c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.250360 May 25 16:19:38.191326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 160/330] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:37 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:38.205975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a7806fe6-a780-4210-bf4c-21c64add2192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:38.207047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a7806fe6-a780-4210-bf4c-21c64add2192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:38.214964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a7806fe6-a780-4210-bf4c-21c64add2192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.215755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a7806fe6-a780-4210-bf4c-21c64add2192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.216700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a7806fe6-a780-4210-bf4c-21c64add2192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.217276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a7806fe6-a780-4210-bf4c-21c64add2192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.354040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ba14bdf4-b91a-4866-a1ff-fa9c9119f0e8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:38.367076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ba14bdf4-b91a-4866-a1ff-fa9c9119f0e8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.365663 May 25 16:19:38.378007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 171/331] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:38 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:38.383866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-645cdbf5-185d-4218-adc7-1a036a7dbaef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'create', calling method: .version_select at 0x7f4a30021160>, body: {"keypair": {"name": "tempest-keypair-1809206957", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHRX7nPldvVXy/wm3RvwcIl9WXSoOL4vGnGl3xjmXTH47qYtsHNX7oxDSMQQVrZTR9Rt3kf/ZEtiHNsdw+9vtsAu/8NdqvB5d0qsIdegHd4Y8YA+F42MM4c7iFSFkexNng=="}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:38.392923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a7806fe6-a780-4210-bf4c-21c64add2192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:38.404890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-645cdbf5-185d-4218-adc7-1a036a7dbaef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Getting quotas for project bd0c9fdb2e8e4e72ac2835b0482ae824. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:19:38.405597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a7806fe6-a780-4210-bf4c-21c64add2192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.212233 May 25 16:19:38.406559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 161/332] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:38 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:38.411212 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-645cdbf5-185d-4218-adc7-1a036a7dbaef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Getting quotas for user 829c30cd914f48548b987bd6e39ee774 and project bd0c9fdb2e8e4e72ac2835b0482ae824. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:19:38.411958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:38.412580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:38.420510 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.424902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.427214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.427865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.440376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-645cdbf5-185d-4218-adc7-1a036a7dbaef tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.060735 May 25 16:19:38.441252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 172/333] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:19:38 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:38.469266 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-41bbf7d1-b1ce-40e2-a413-588faf52dbba tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:38.469885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-41bbf7d1-b1ce-40e2-a413-588faf52dbba tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:38.501964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41bbf7d1-b1ce-40e2-a413-588faf52dbba tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.509155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41bbf7d1-b1ce-40e2-a413-588faf52dbba tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.510284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41bbf7d1-b1ce-40e2-a413-588faf52dbba tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.512722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41bbf7d1-b1ce-40e2-a413-588faf52dbba tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.606475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-41bbf7d1-b1ce-40e2-a413-588faf52dbba tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:38.616665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-41bbf7d1-b1ce-40e2-a413-588faf52dbba tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1223 microversion: 2.1 time: 0.173982 May 25 16:19:38.617249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 173/334] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:38 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1223 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:38.622284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fa97c2f9-6385-4b4d-a2a7-57c91a178340 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:38.623055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fa97c2f9-6385-4b4d-a2a7-57c91a178340 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:38.629475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa97c2f9-6385-4b4d-a2a7-57c91a178340 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.630389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa97c2f9-6385-4b4d-a2a7-57c91a178340 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.636696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa97c2f9-6385-4b4d-a2a7-57c91a178340 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:38.637560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa97c2f9-6385-4b4d-a2a7-57c91a178340 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:38.656798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:38.669473 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:38.671445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:38.672938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9edb6b3f-f288-405a-a74c-2501e3345e56 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.265205 May 25 16:19:38.673553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 162/335] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:38 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:38.773559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fa97c2f9-6385-4b4d-a2a7-57c91a178340 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:38.782871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fa97c2f9-6385-4b4d-a2a7-57c91a178340 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1245 microversion: 2.1 time: 0.164410 May 25 16:19:38.784040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 174/336] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:38 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1245 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:39.197466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c3ec199a-02fb-4a29-93e3-9925f7dedeba tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:39.198137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c3ec199a-02fb-4a29-93e3-9925f7dedeba tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:39.206636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3ec199a-02fb-4a29-93e3-9925f7dedeba tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.207693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3ec199a-02fb-4a29-93e3-9925f7dedeba tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.208744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3ec199a-02fb-4a29-93e3-9925f7dedeba tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.209287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3ec199a-02fb-4a29-93e3-9925f7dedeba tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.385785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7187eeca-abe8-448a-beec-d19f3d6a5862 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:39.386969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7187eeca-abe8-448a-beec-d19f3d6a5862 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:39.396393 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c3ec199a-02fb-4a29-93e3-9925f7dedeba tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:39.397214 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7187eeca-abe8-448a-beec-d19f3d6a5862 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.399587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7187eeca-abe8-448a-beec-d19f3d6a5862 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.400510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7187eeca-abe8-448a-beec-d19f3d6a5862 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.400989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7187eeca-abe8-448a-beec-d19f3d6a5862 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.405788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c3ec199a-02fb-4a29-93e3-9925f7dedeba tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.213439 May 25 16:19:39.407511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 163/337] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:39 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:39.428760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c1df4151-fc17-4980-8d4c-ff1a0833fd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:39.430680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c1df4151-fc17-4980-8d4c-ff1a0833fd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:39.441116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1df4151-fc17-4980-8d4c-ff1a0833fd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.441707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1df4151-fc17-4980-8d4c-ff1a0833fd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.442764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1df4151-fc17-4980-8d4c-ff1a0833fd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.443323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1df4151-fc17-4980-8d4c-ff1a0833fd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.605961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7187eeca-abe8-448a-beec-d19f3d6a5862 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:39.615216 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7187eeca-abe8-448a-beec-d19f3d6a5862 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.233973 May 25 16:19:39.616185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 175/338] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:39 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:39.627950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c1df4151-fc17-4980-8d4c-ff1a0833fd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:39.633923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-df5d5472-7939-4d9d-ad77-1a4ab748c3ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:39.634755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-df5d5472-7939-4d9d-ad77-1a4ab748c3ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:39.638933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c1df4151-fc17-4980-8d4c-ff1a0833fd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.217142 May 25 16:19:39.647139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 164/339] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:39 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:39.649326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df5d5472-7939-4d9d-ad77-1a4ab748c3ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.649800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df5d5472-7939-4d9d-ad77-1a4ab748c3ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.650582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df5d5472-7939-4d9d-ad77-1a4ab748c3ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.651019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df5d5472-7939-4d9d-ad77-1a4ab748c3ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.691185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:39.691776 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:39.698474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.698951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.699828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.700260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.721038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-df5d5472-7939-4d9d-ad77-1a4ab748c3ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:39.730021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-df5d5472-7939-4d9d-ad77-1a4ab748c3ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1271 microversion: 2.1 time: 0.100218 May 25 16:19:39.745243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 176/340] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:39 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1271 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:39.805412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2355f7a4-7510-4e51-accd-454149605c8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:39.806346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2355f7a4-7510-4e51-accd-454149605c8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:39.815510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2355f7a4-7510-4e51-accd-454149605c8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.816154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2355f7a4-7510-4e51-accd-454149605c8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.818599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2355f7a4-7510-4e51-accd-454149605c8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:39.819305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2355f7a4-7510-4e51-accd-454149605c8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:39.946571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:39.951527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2355f7a4-7510-4e51-accd-454149605c8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:39.956914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:39.958612 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:39.960147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2355f7a4-7510-4e51-accd-454149605c8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1293 microversion: 2.1 time: 0.162867 May 25 16:19:39.960287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-12e947a5-0417-4c50-8dc1-c69ffe0f5615 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.272610 May 25 16:19:39.960454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 177/341] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:39 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1293 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:39.969606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 165/342] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:39 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:39.982150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1683999046", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "729d4051-3bd3-41a9-b052-4f07a89919be"}], "security_groups": [{"name": "tempest-securitygroup--1781579868"}], "key_name": "tempest-keypair-1809206957", "user_data": "}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:40.087018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.network.neutron [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] validate_networks() for [('729d4051-3bd3-41a9-b052-4f07a89919be', None, None, None, None, None)] {{(pid=117794) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:19:40.426247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6e2e98e9-1eda-4f0c-8122-9b0a21552fcd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:40.427026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6e2e98e9-1eda-4f0c-8122-9b0a21552fcd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:40.434584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e2e98e9-1eda-4f0c-8122-9b0a21552fcd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:40.435287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e2e98e9-1eda-4f0c-8122-9b0a21552fcd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:40.436142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e2e98e9-1eda-4f0c-8122-9b0a21552fcd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:40.436633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e2e98e9-1eda-4f0c-8122-9b0a21552fcd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:40.934614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6e2e98e9-1eda-4f0c-8122-9b0a21552fcd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:40.944511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6e2e98e9-1eda-4f0c-8122-9b0a21552fcd tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1265 microversion: 2.1 time: 0.522191 May 25 16:19:40.945359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 166/343] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:40 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1265 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:40.950614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fa80dfe3-1cc0-4b9b-af1d-583adaad1ee2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:40.951258 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fa80dfe3-1cc0-4b9b-af1d-583adaad1ee2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:40.958057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fa80dfe3-1cc0-4b9b-af1d-583adaad1ee2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:40.958509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fa80dfe3-1cc0-4b9b-af1d-583adaad1ee2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:40.959304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fa80dfe3-1cc0-4b9b-af1d-583adaad1ee2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:40.959733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fa80dfe3-1cc0-4b9b-af1d-583adaad1ee2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.133297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Flavor limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:19:41.134382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Image limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:19:41.135025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Flavor pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:19:41.135708 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Image pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:19:41.136501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:19:41.148982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.149762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.151004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.152453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.171630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Getting quotas for project bd0c9fdb2e8e4e72ac2835b0482ae824. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:19:41.175666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Getting quotas for user 829c30cd914f48548b987bd6e39ee774 and project bd0c9fdb2e8e4e72ac2835b0482ae824. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:19:41.187295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Going to run 1 instances... {{(pid=117794) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:19:41.189622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117794) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:19:41.257989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c8540c21-6b88-41bc-a93e-4c8c31a025fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.273534 May 25 16:19:41.263390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 178/344] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:19:39 2022] POST /compute/v2.1/servers => generated 409 bytes in 1287 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:19:41.302138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b1822f7e-e9c4-4a84-b680-8594d19b94de tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:41.302832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b1822f7e-e9c4-4a84-b680-8594d19b94de tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:41.309459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b1822f7e-e9c4-4a84-b680-8594d19b94de tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.310132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b1822f7e-e9c4-4a84-b680-8594d19b94de tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.311057 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b1822f7e-e9c4-4a84-b680-8594d19b94de tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.311659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b1822f7e-e9c4-4a84-b680-8594d19b94de tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.314282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fa80dfe3-1cc0-4b9b-af1d-583adaad1ee2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:41.332673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fa80dfe3-1cc0-4b9b-af1d-583adaad1ee2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.385904 May 25 16:19:41.333701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 167/345] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:40 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:41.340240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-574f17c4-a86e-40fa-be8f-2bb3d3a75c18 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:41.341032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-574f17c4-a86e-40fa-be8f-2bb3d3a75c18 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:41.354439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-574f17c4-a86e-40fa-be8f-2bb3d3a75c18 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.355240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-574f17c4-a86e-40fa-be8f-2bb3d3a75c18 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.356185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-574f17c4-a86e-40fa-be8f-2bb3d3a75c18 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.356771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-574f17c4-a86e-40fa-be8f-2bb3d3a75c18 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.551240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-574f17c4-a86e-40fa-be8f-2bb3d3a75c18 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:41.572304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-574f17c4-a86e-40fa-be8f-2bb3d3a75c18 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1277 microversion: 2.1 time: 0.237269 May 25 16:19:41.575996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 168/346] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:41 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1277 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:41.586426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-efc3ccad-badf-4f56-bff6-0f0417dbe725 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:41.587352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-efc3ccad-badf-4f56-bff6-0f0417dbe725 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:41.594630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efc3ccad-badf-4f56-bff6-0f0417dbe725 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.595542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efc3ccad-badf-4f56-bff6-0f0417dbe725 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.596475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efc3ccad-badf-4f56-bff6-0f0417dbe725 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.597083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efc3ccad-badf-4f56-bff6-0f0417dbe725 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.735654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-efc3ccad-badf-4f56-bff6-0f0417dbe725 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:41.747207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-efc3ccad-badf-4f56-bff6-0f0417dbe725 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1299 microversion: 2.1 time: 0.169864 May 25 16:19:41.748436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 169/347] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:41 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1299 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:41.770490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:41.771116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:41.787189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.788004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.789875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.790529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.878754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b1822f7e-e9c4-4a84-b680-8594d19b94de tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:41.897421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b1822f7e-e9c4-4a84-b680-8594d19b94de tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.623991 May 25 16:19:41.898004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 179/348] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:41 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:41.906687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c1cb15db-9b30-40e4-ac52-fca64ace26de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:41.907773 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c1cb15db-9b30-40e4-ac52-fca64ace26de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:41.915560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c1cb15db-9b30-40e4-ac52-fca64ace26de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.917962 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c1cb15db-9b30-40e4-ac52-fca64ace26de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.918781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c1cb15db-9b30-40e4-ac52-fca64ace26de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:41.919856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c1cb15db-9b30-40e4-ac52-fca64ace26de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:41.925536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:42.087687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c1cb15db-9b30-40e4-ac52-fca64ace26de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:42.096924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c1cb15db-9b30-40e4-ac52-fca64ace26de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1242 microversion: 2.1 time: 0.195916 May 25 16:19:42.097815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 180/349] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:41 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1242 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:42.102828 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6b0a9d82-dc42-4687-b387-072563755ca3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:42.103539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6b0a9d82-dc42-4687-b387-072563755ca3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:42.111065 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b0a9d82-dc42-4687-b387-072563755ca3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.111560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b0a9d82-dc42-4687-b387-072563755ca3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.112862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b0a9d82-dc42-4687-b387-072563755ca3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.113604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b0a9d82-dc42-4687-b387-072563755ca3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.224814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:42.233428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:42.234929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:42.236268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-277b67df-25df-48bf-bb21-15d4165bef0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.486515 May 25 16:19:42.236973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 170/350] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:42.351331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1d1942c2-e0ec-4e42-a0b8-cb0783e27cef tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:42.353161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1d1942c2-e0ec-4e42-a0b8-cb0783e27cef tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:42.376238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d1942c2-e0ec-4e42-a0b8-cb0783e27cef tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.379263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d1942c2-e0ec-4e42-a0b8-cb0783e27cef tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.381247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d1942c2-e0ec-4e42-a0b8-cb0783e27cef tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.381853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d1942c2-e0ec-4e42-a0b8-cb0783e27cef tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.449625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6b0a9d82-dc42-4687-b387-072563755ca3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:42.463879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6b0a9d82-dc42-4687-b387-072563755ca3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1452 microversion: 2.1 time: 0.364693 May 25 16:19:42.464654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 181/351] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:42 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1452 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:42.481866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-74c7914a-ba19-4db9-8a7c-f32977f089c6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:42.482612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-74c7914a-ba19-4db9-8a7c-f32977f089c6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:42.488743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-74c7914a-ba19-4db9-8a7c-f32977f089c6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.489383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-74c7914a-ba19-4db9-8a7c-f32977f089c6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.490820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-74c7914a-ba19-4db9-8a7c-f32977f089c6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.491411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-74c7914a-ba19-4db9-8a7c-f32977f089c6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.833278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d1942c2-e0ec-4e42-a0b8-cb0783e27cef tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:42.841977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1d1942c2-e0ec-4e42-a0b8-cb0783e27cef tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.495979 May 25 16:19:42.842555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 171/352] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:42 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:42.847874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7a1e26ca-ce29-42a5-b094-8294e6d0d696 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:42.848921 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7a1e26ca-ce29-42a5-b094-8294e6d0d696 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:42.856452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1e26ca-ce29-42a5-b094-8294e6d0d696 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.857025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1e26ca-ce29-42a5-b094-8294e6d0d696 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.857787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1e26ca-ce29-42a5-b094-8294e6d0d696 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.858010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-74c7914a-ba19-4db9-8a7c-f32977f089c6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:42.858213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1e26ca-ce29-42a5-b094-8294e6d0d696 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.868314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-74c7914a-ba19-4db9-8a7c-f32977f089c6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1452 microversion: 2.1 time: 0.390514 May 25 16:19:42.869630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 182/353] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:42 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1452 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:42.879192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-50582d0e-fcd7-48a3-99ef-b9986bd81b46 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:42.880658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-50582d0e-fcd7-48a3-99ef-b9986bd81b46 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:42.896692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50582d0e-fcd7-48a3-99ef-b9986bd81b46 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.897278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50582d0e-fcd7-48a3-99ef-b9986bd81b46 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.898529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50582d0e-fcd7-48a3-99ef-b9986bd81b46 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:42.899180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50582d0e-fcd7-48a3-99ef-b9986bd81b46 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:42.899467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:43.635265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7a1e26ca-ce29-42a5-b094-8294e6d0d696 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:43.651698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7a1e26ca-ce29-42a5-b094-8294e6d0d696 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.807643 May 25 16:19:43.652829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 172/354] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:42 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 809 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:43.657646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2740023d-7c83-4b95-a21a-9fa44146b1d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:43.658313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2740023d-7c83-4b95-a21a-9fa44146b1d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:43.666438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2740023d-7c83-4b95-a21a-9fa44146b1d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:43.666932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2740023d-7c83-4b95-a21a-9fa44146b1d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:43.667964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2740023d-7c83-4b95-a21a-9fa44146b1d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:43.669136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2740023d-7c83-4b95-a21a-9fa44146b1d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:43.674202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-50582d0e-fcd7-48a3-99ef-b9986bd81b46 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:43.686891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-50582d0e-fcd7-48a3-99ef-b9986bd81b46 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.814712 May 25 16:19:43.687721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 183/355] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:42 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 818 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:43.693982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9fcc601c-e467-4b82-b3d6-f771560bcd81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:43.694862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9fcc601c-e467-4b82-b3d6-f771560bcd81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:43.705685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9fcc601c-e467-4b82-b3d6-f771560bcd81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:43.706581 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9fcc601c-e467-4b82-b3d6-f771560bcd81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:43.707675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9fcc601c-e467-4b82-b3d6-f771560bcd81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:43.708480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9fcc601c-e467-4b82-b3d6-f771560bcd81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:43.834017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9fcc601c-e467-4b82-b3d6-f771560bcd81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:43.863182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9fcc601c-e467-4b82-b3d6-f771560bcd81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1304 microversion: 2.1 time: 0.163447 May 25 16:19:43.863182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 184/356] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:43 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1304 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:43.871620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:43.871620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:43.890691 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:43.891274 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:43.892122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:43.892793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:44.096482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2740023d-7c83-4b95-a21a-9fa44146b1d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:44.108940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2740023d-7c83-4b95-a21a-9fa44146b1d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.454830 May 25 16:19:44.118856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 173/357] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:43 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:44.118856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a706c178-1499-4127-89a0-9ba0ae67fcf7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:44.118856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a706c178-1499-4127-89a0-9ba0ae67fcf7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:44.132746 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a706c178-1499-4127-89a0-9ba0ae67fcf7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:44.134161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a706c178-1499-4127-89a0-9ba0ae67fcf7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:44.135077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a706c178-1499-4127-89a0-9ba0ae67fcf7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:44.137440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a706c178-1499-4127-89a0-9ba0ae67fcf7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:44.412927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:44.426487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:44.428350 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:44.429793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1108efab-7556-42f5-9346-a2d4bab18066 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1724 microversion: 2.20 time: 0.572646 May 25 16:19:44.431951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 185/358] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1724 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:44.640734 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a706c178-1499-4127-89a0-9ba0ae67fcf7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:44.650145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a706c178-1499-4127-89a0-9ba0ae67fcf7 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1724 microversion: 2.1 time: 0.538992 May 25 16:19:44.650905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 174/359] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:44 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1724 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:44.671319 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2a251a05-8a45-470a-b81f-c81da4b743f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:44.672180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2a251a05-8a45-470a-b81f-c81da4b743f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:44.679640 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a251a05-8a45-470a-b81f-c81da4b743f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:44.680303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a251a05-8a45-470a-b81f-c81da4b743f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:44.680952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a251a05-8a45-470a-b81f-c81da4b743f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:44.681455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a251a05-8a45-470a-b81f-c81da4b743f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:44.704593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-15277391-5edd-480b-a580-c177cdf1ea05 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:44.705587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-15277391-5edd-480b-a580-c177cdf1ea05 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:44.712083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15277391-5edd-480b-a580-c177cdf1ea05 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:44.712788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15277391-5edd-480b-a580-c177cdf1ea05 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:44.713769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15277391-5edd-480b-a580-c177cdf1ea05 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:44.714206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15277391-5edd-480b-a580-c177cdf1ea05 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.095868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-15277391-5edd-480b-a580-c177cdf1ea05 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:45.106498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2a251a05-8a45-470a-b81f-c81da4b743f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:45.106788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-15277391-5edd-480b-a580-c177cdf1ea05 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.405919 May 25 16:19:45.120586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 175/360] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:44 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:45.122665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2a251a05-8a45-470a-b81f-c81da4b743f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.452415 May 25 16:19:45.124685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 186/361] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:44 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:45.127053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-59ef3863-6d49-4f95-9b58-1b830b569b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:45.128417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-59ef3863-6d49-4f95-9b58-1b830b569b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:45.137720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7908a99b-aae1-4f9c-adf2-34d3b9b200a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:45.138434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7908a99b-aae1-4f9c-adf2-34d3b9b200a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:45.139589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59ef3863-6d49-4f95-9b58-1b830b569b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.140052 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59ef3863-6d49-4f95-9b58-1b830b569b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.140852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59ef3863-6d49-4f95-9b58-1b830b569b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.144097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59ef3863-6d49-4f95-9b58-1b830b569b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.148418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7908a99b-aae1-4f9c-adf2-34d3b9b200a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.148886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7908a99b-aae1-4f9c-adf2-34d3b9b200a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.149827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7908a99b-aae1-4f9c-adf2-34d3b9b200a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.150263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7908a99b-aae1-4f9c-adf2-34d3b9b200a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.278618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-59ef3863-6d49-4f95-9b58-1b830b569b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:45.291909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-59ef3863-6d49-4f95-9b58-1b830b569b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1304 microversion: 2.1 time: 0.168410 May 25 16:19:45.293517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 176/362] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:45 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1304 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:45.300187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-551340c3-26c6-41fd-afe0-49324e45774e req-f9e5ed2e-5fb4-45b1-b485-3fcdaae2c036 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b3b920a7-fa28-4150-b889-d9a2ea63bc16", "tag": "533fafe5-1581-4d83-887b-681b728ba38c"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:45.312914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-551340c3-26c6-41fd-afe0-49324e45774e req-f9e5ed2e-5fb4-45b1-b485-3fcdaae2c036 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.313933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-551340c3-26c6-41fd-afe0-49324e45774e req-f9e5ed2e-5fb4-45b1-b485-3fcdaae2c036 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.327375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-551340c3-26c6-41fd-afe0-49324e45774e req-f9e5ed2e-5fb4-45b1-b485-3fcdaae2c036 service nova] Creating event network-changed:533fafe5-1581-4d83-887b-681b728ba38c for instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 on hv-cinder-84327 May 25 16:19:45.333656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-551340c3-26c6-41fd-afe0-49324e45774e req-f9e5ed2e-5fb4-45b1-b485-3fcdaae2c036 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038613 May 25 16:19:45.334616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 177/363] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:45.453143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:45.455346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:45.470403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.471243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.474136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.474572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.631415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7908a99b-aae1-4f9c-adf2-34d3b9b200a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:45.641930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7908a99b-aae1-4f9c-adf2-34d3b9b200a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.513402 May 25 16:19:45.643190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 187/364] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:45 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:45.669271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-52fc7b1e-d39a-4f00-b8d2-d4238a747237 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:45.670091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-52fc7b1e-d39a-4f00-b8d2-d4238a747237 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:45.677831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-52fc7b1e-d39a-4f00-b8d2-d4238a747237 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.678318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-52fc7b1e-d39a-4f00-b8d2-d4238a747237 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.679091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-52fc7b1e-d39a-4f00-b8d2-d4238a747237 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.679551 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-52fc7b1e-d39a-4f00-b8d2-d4238a747237 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:45.943506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:45.952566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:45.954244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:45.955613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9205da10-fc36-4dc3-88be-f4d21807bcfb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1745 microversion: 2.20 time: 0.509004 May 25 16:19:45.956272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 178/365] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:45 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1745 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:45.989329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-01bd147f-bf31-4551-ab17-b44e74f5dda0 req-68d00a06-b4ff-45a8-9e9c-1c56905d83b0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "516581ca-a13c-4a28-92da-988be630b29c", "tag": "798e662c-998a-4c12-afb9-3d2641d4ff07"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:45.998046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-01bd147f-bf31-4551-ab17-b44e74f5dda0 req-68d00a06-b4ff-45a8-9e9c-1c56905d83b0 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:45.998889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-01bd147f-bf31-4551-ab17-b44e74f5dda0 req-68d00a06-b4ff-45a8-9e9c-1c56905d83b0 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:46.010446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-01bd147f-bf31-4551-ab17-b44e74f5dda0 req-68d00a06-b4ff-45a8-9e9c-1c56905d83b0 service nova] Creating event network-changed:798e662c-998a-4c12-afb9-3d2641d4ff07 for instance 516581ca-a13c-4a28-92da-988be630b29c on hv-cinder-84327 May 25 16:19:46.015433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-01bd147f-bf31-4551-ab17-b44e74f5dda0 req-68d00a06-b4ff-45a8-9e9c-1c56905d83b0 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030214 May 25 16:19:46.016368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 179/366] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:46.107871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-52fc7b1e-d39a-4f00-b8d2-d4238a747237 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:46.119309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-52fc7b1e-d39a-4f00-b8d2-d4238a747237 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1701 microversion: 2.1 time: 0.454721 May 25 16:19:46.120535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 188/367] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:45 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1701 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:46.132120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bd9ebd77-137c-4493-bb93-223896356508 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:46.133531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bd9ebd77-137c-4493-bb93-223896356508 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:46.141847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fa831631-e9ed-4472-b942-a0478f7c8adb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:46.144293 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd9ebd77-137c-4493-bb93-223896356508 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:46.145159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd9ebd77-137c-4493-bb93-223896356508 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:46.148249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd9ebd77-137c-4493-bb93-223896356508 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:46.148697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd9ebd77-137c-4493-bb93-223896356508 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:46.158848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa831631-e9ed-4472-b942-a0478f7c8adb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:46.159313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa831631-e9ed-4472-b942-a0478f7c8adb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:46.224583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fa831631-e9ed-4472-b942-a0478f7c8adb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Going to try to start instance {{(pid=117794) start /opt/stack/nova/nova/compute/api.py:2730}} May 25 16:19:46.281712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fa831631-e9ed-4472-b942-a0478f7c8adb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/action" status: 202 len: 0 microversion: 2.1 time: 0.148727 May 25 16:19:46.283511 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 189/368] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:19:46 2022] POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/action => generated 0 bytes in 152 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:19:46.295127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4fcb29fb-b363-47c6-a996-86e46fe63ed1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:46.295911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4fcb29fb-b363-47c6-a996-86e46fe63ed1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:46.302486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fcb29fb-b363-47c6-a996-86e46fe63ed1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:46.304405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fcb29fb-b363-47c6-a996-86e46fe63ed1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:46.304653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fcb29fb-b363-47c6-a996-86e46fe63ed1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:46.305426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fcb29fb-b363-47c6-a996-86e46fe63ed1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:46.771441 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bd9ebd77-137c-4493-bb93-223896356508 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:46.779435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bd9ebd77-137c-4493-bb93-223896356508 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.653189 May 25 16:19:46.780015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 180/369] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:46 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:46.784904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a8611e6c-17e4-4a4a-847a-46f4966c8875 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:46.785761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a8611e6c-17e4-4a4a-847a-46f4966c8875 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:46.792459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8611e6c-17e4-4a4a-847a-46f4966c8875 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:46.793013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8611e6c-17e4-4a4a-847a-46f4966c8875 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:46.793841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8611e6c-17e4-4a4a-847a-46f4966c8875 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:46.794338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8611e6c-17e4-4a4a-847a-46f4966c8875 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:46.813791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4fcb29fb-b363-47c6-a996-86e46fe63ed1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:46.821769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4fcb29fb-b363-47c6-a996-86e46fe63ed1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.536167 May 25 16:19:46.822523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 190/370] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:46 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:46.827360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-66f7bc67-0b06-4ab2-8b9f-06dc809904c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:46.828038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-66f7bc67-0b06-4ab2-8b9f-06dc809904c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:46.835237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66f7bc67-0b06-4ab2-8b9f-06dc809904c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:46.835688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66f7bc67-0b06-4ab2-8b9f-06dc809904c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:46.836458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66f7bc67-0b06-4ab2-8b9f-06dc809904c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:46.836888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66f7bc67-0b06-4ab2-8b9f-06dc809904c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:47.226809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a8611e6c-17e4-4a4a-847a-46f4966c8875 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:47.236564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a8611e6c-17e4-4a4a-847a-46f4966c8875 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1710 microversion: 2.1 time: 0.455543 May 25 16:19:47.246332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 181/371] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:46 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1710 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:47.251451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f372c151-69b2-437e-b80f-73f0f6096463 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:47.252007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f372c151-69b2-437e-b80f-73f0f6096463 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:47.269450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f372c151-69b2-437e-b80f-73f0f6096463 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:47.269750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f372c151-69b2-437e-b80f-73f0f6096463 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:47.270587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f372c151-69b2-437e-b80f-73f0f6096463 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:47.271022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f372c151-69b2-437e-b80f-73f0f6096463 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:47.379700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-66f7bc67-0b06-4ab2-8b9f-06dc809904c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:47.392328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-66f7bc67-0b06-4ab2-8b9f-06dc809904c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.568126 May 25 16:19:47.393931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 191/372] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:46 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:47.401750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:47.402918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:47.412578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:47.413909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:47.414993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:47.415648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:47.847337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:47.869128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=6 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:47.902729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f372c151-69b2-437e-b80f-73f0f6096463 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:47.911267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f372c151-69b2-437e-b80f-73f0f6096463 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.667376 May 25 16:19:47.911827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 182/373] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:47 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:47.916795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9570b58d-e032-4090-9027-1297a4b1e0a2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:47.917414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9570b58d-e032-4090-9027-1297a4b1e0a2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:47.923520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9570b58d-e032-4090-9027-1297a4b1e0a2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:47.923939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9570b58d-e032-4090-9027-1297a4b1e0a2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:47.924738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9570b58d-e032-4090-9027-1297a4b1e0a2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:47.925170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9570b58d-e032-4090-9027-1297a4b1e0a2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:47.962502 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:47.971801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:47.973299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:47.974532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-19763295-6f20-4a52-8d87-87f8258dfacb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1745 microversion: 2.20 time: 0.579449 May 25 16:19:47.975094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 192/374] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1745 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:47.979525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c6fae167-969b-4486-b3c5-65ade5ed4b5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:47.980144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c6fae167-969b-4486-b3c5-65ade5ed4b5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:47.986340 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6fae167-969b-4486-b3c5-65ade5ed4b5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:47.986785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6fae167-969b-4486-b3c5-65ade5ed4b5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:47.987577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6fae167-969b-4486-b3c5-65ade5ed4b5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:47.988009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6fae167-969b-4486-b3c5-65ade5ed4b5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:48.662405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9570b58d-e032-4090-9027-1297a4b1e0a2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:48.670427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9570b58d-e032-4090-9027-1297a4b1e0a2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.757539 May 25 16:19:48.670924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 183/375] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:47 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:48.675356 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-d8111408-b536-4d4b-a8a3-f4002c015482 req-15126068-7edd-4fb6-8b9a-74f3717f7f21 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "667322c4-d9ca-4320-903f-112d84e5f462", "tag": "0906f7f3-63c4-42d4-9006-1ed37e306e92"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:48.684601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-d8111408-b536-4d4b-a8a3-f4002c015482 req-15126068-7edd-4fb6-8b9a-74f3717f7f21 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:48.685043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-d8111408-b536-4d4b-a8a3-f4002c015482 req-15126068-7edd-4fb6-8b9a-74f3717f7f21 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:48.696565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-d8111408-b536-4d4b-a8a3-f4002c015482 req-15126068-7edd-4fb6-8b9a-74f3717f7f21 service nova] Creating event network-changed:0906f7f3-63c4-42d4-9006-1ed37e306e92 for instance 667322c4-d9ca-4320-903f-112d84e5f462 on hv-cinder-84327 May 25 16:19:48.701213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-d8111408-b536-4d4b-a8a3-f4002c015482 req-15126068-7edd-4fb6-8b9a-74f3717f7f21 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029141 May 25 16:19:48.701802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 184/376] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:48.706679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-03ba2f60-63e0-4de1-aad5-7c4aba8540c2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:48.707397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-03ba2f60-63e0-4de1-aad5-7c4aba8540c2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:48.713118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c6fae167-969b-4486-b3c5-65ade5ed4b5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:48.713593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03ba2f60-63e0-4de1-aad5-7c4aba8540c2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:48.713960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03ba2f60-63e0-4de1-aad5-7c4aba8540c2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:48.714717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03ba2f60-63e0-4de1-aad5-7c4aba8540c2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:48.715147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03ba2f60-63e0-4de1-aad5-7c4aba8540c2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:48.721728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c6fae167-969b-4486-b3c5-65ade5ed4b5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.745579 May 25 16:19:48.722862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 193/377] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:47 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 747 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:48.728781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-65b881fd-f99b-479b-829c-b392d2bc67f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:48.729465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-65b881fd-f99b-479b-829c-b392d2bc67f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:48.735977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-65b881fd-f99b-479b-829c-b392d2bc67f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:48.736688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-65b881fd-f99b-479b-829c-b392d2bc67f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:48.737550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-65b881fd-f99b-479b-829c-b392d2bc67f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:48.738073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-65b881fd-f99b-479b-829c-b392d2bc67f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:49.014544 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-03ba2f60-63e0-4de1-aad5-7c4aba8540c2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:49.023664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-03ba2f60-63e0-4de1-aad5-7c4aba8540c2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1714 microversion: 2.1 time: 0.320636 May 25 16:19:49.036489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 185/378] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:48 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1714 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:49.048067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f9b312f3-9f65-4a4e-b4cf-03656cee96c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:49.050093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f9b312f3-9f65-4a4e-b4cf-03656cee96c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:49.054482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-65b881fd-f99b-479b-829c-b392d2bc67f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:49.059384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f9b312f3-9f65-4a4e-b4cf-03656cee96c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:49.060089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f9b312f3-9f65-4a4e-b4cf-03656cee96c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:49.061074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f9b312f3-9f65-4a4e-b4cf-03656cee96c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:49.061721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f9b312f3-9f65-4a4e-b4cf-03656cee96c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:49.068120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-65b881fd-f99b-479b-829c-b392d2bc67f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.343164 May 25 16:19:49.068961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 194/379] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:48 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:49.084287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:49.086028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:49.104943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:49.105583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:49.106629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:49.107147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:49.567914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f9b312f3-9f65-4a4e-b4cf-03656cee96c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:49.579273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f9b312f3-9f65-4a4e-b4cf-03656cee96c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.541237 May 25 16:19:49.580986 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 186/380] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:49 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:49.586320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-6496cf28-664c-425e-a6c7-64e18711bcea req-45a0bf68-e80f-4208-ace4-f2aa71e6735e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d86354dc-f3d4-409d-b331-9f708947be2a", "name": "network-vif-plugged", "status": "completed", "tag": "e2c0938e-f9fb-4796-87fa-103cbcfd2536"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:49.609682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-6496cf28-664c-425e-a6c7-64e18711bcea req-45a0bf68-e80f-4208-ace4-f2aa71e6735e service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:49.610572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-6496cf28-664c-425e-a6c7-64e18711bcea req-45a0bf68-e80f-4208-ace4-f2aa71e6735e service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:49.631498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:49.634795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:49.644857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-6496cf28-664c-425e-a6c7-64e18711bcea req-45a0bf68-e80f-4208-ace4-f2aa71e6735e service nova] Creating event network-vif-plugged:e2c0938e-f9fb-4796-87fa-103cbcfd2536 for instance d86354dc-f3d4-409d-b331-9f708947be2a on hv-cinder-84327 May 25 16:19:49.645667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:49.649089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:49.651405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3a7245de-1d27-4772-a86a-a1ceee4f1fa9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1745 microversion: 2.20 time: 0.580460 May 25 16:19:49.652674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 195/381] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1745 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:49.653767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-6496cf28-664c-425e-a6c7-64e18711bcea req-45a0bf68-e80f-4208-ace4-f2aa71e6735e service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071352 May 25 16:19:49.654587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 187/382] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:49.694069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d44d0c95-576c-465c-a031-06671dcba80c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:49.695529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d44d0c95-576c-465c-a031-06671dcba80c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:49.706841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d44d0c95-576c-465c-a031-06671dcba80c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:49.707759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d44d0c95-576c-465c-a031-06671dcba80c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:49.709238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d44d0c95-576c-465c-a031-06671dcba80c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:49.709898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d44d0c95-576c-465c-a031-06671dcba80c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:49.742818 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-724a8206-3a8d-4b40-9e27-d8b46b5e4f91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:49.744552 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-724a8206-3a8d-4b40-9e27-d8b46b5e4f91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:49.760916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-724a8206-3a8d-4b40-9e27-d8b46b5e4f91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:49.763367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-724a8206-3a8d-4b40-9e27-d8b46b5e4f91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:49.764942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-724a8206-3a8d-4b40-9e27-d8b46b5e4f91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:49.766013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-724a8206-3a8d-4b40-9e27-d8b46b5e4f91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:50.136403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d44d0c95-576c-465c-a031-06671dcba80c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:50.146945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-724a8206-3a8d-4b40-9e27-d8b46b5e4f91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:50.148358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d44d0c95-576c-465c-a031-06671dcba80c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.462674 May 25 16:19:50.149248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 196/383] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:49 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:50.155228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-205e0008-02ee-4be9-a77e-c7bd953dae3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:50.155488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-724a8206-3a8d-4b40-9e27-d8b46b5e4f91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.418445 May 25 16:19:50.163296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-205e0008-02ee-4be9-a77e-c7bd953dae3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:50.163924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 188/384] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:49 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:50.172817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-205e0008-02ee-4be9-a77e-c7bd953dae3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:50.173271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-205e0008-02ee-4be9-a77e-c7bd953dae3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:50.174039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-205e0008-02ee-4be9-a77e-c7bd953dae3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:50.174640 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-205e0008-02ee-4be9-a77e-c7bd953dae3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:50.359689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-205e0008-02ee-4be9-a77e-c7bd953dae3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:50.367626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-205e0008-02ee-4be9-a77e-c7bd953dae3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.216973 May 25 16:19:50.393035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 197/385] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:50 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:50.603935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c651cff9-d08d-421a-a818-e9760b4a966d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:50.604858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c651cff9-d08d-421a-a818-e9760b4a966d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:50.613366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c651cff9-d08d-421a-a818-e9760b4a966d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:50.614140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c651cff9-d08d-421a-a818-e9760b4a966d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:50.615270 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c651cff9-d08d-421a-a818-e9760b4a966d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:50.615714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c651cff9-d08d-421a-a818-e9760b4a966d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:50.682315 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:50.683292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:50.696849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:50.699405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:50.701790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:50.702659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:51.149704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c651cff9-d08d-421a-a818-e9760b4a966d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:51.162181 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=4 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:51.170247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c651cff9-d08d-421a-a818-e9760b4a966d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.574006 May 25 16:19:51.185026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=16 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:51.186317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 189/386] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:50 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:51.202651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:51.217716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:51.218470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-b00df954-5836-4618-bbe4-3ed4cb96ebe5 req-5e2b8d87-a1e3-4eda-a5ff-9962cc41c27b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "621e54b1-cdde-493f-9bad-1c1222a0731a", "tag": "9b36da50-ea60-4505-bbbd-48a647e0698b"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:51.228804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:51.233261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a6e2b4f3-c790-4cce-aa27-aaffe6898eca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1740 microversion: 2.20 time: 0.556149 May 25 16:19:51.241703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 198/387] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:50 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1740 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:51.248540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-b00df954-5836-4618-bbe4-3ed4cb96ebe5 req-5e2b8d87-a1e3-4eda-a5ff-9962cc41c27b service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:51.259255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6427f555-9c68-4472-9cb2-f0d14a717a9f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:51.260211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-b00df954-5836-4618-bbe4-3ed4cb96ebe5 req-5e2b8d87-a1e3-4eda-a5ff-9962cc41c27b service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:51.263638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6427f555-9c68-4472-9cb2-f0d14a717a9f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:51.288207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6427f555-9c68-4472-9cb2-f0d14a717a9f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:51.292790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6427f555-9c68-4472-9cb2-f0d14a717a9f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:51.295713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6427f555-9c68-4472-9cb2-f0d14a717a9f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:51.298162 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-b00df954-5836-4618-bbe4-3ed4cb96ebe5 req-5e2b8d87-a1e3-4eda-a5ff-9962cc41c27b service nova] Creating event network-changed:9b36da50-ea60-4505-bbbd-48a647e0698b for instance 621e54b1-cdde-493f-9bad-1c1222a0731a on hv-cinder-84327 May 25 16:19:51.298849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6427f555-9c68-4472-9cb2-f0d14a717a9f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:51.316686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-b00df954-5836-4618-bbe4-3ed4cb96ebe5 req-5e2b8d87-a1e3-4eda-a5ff-9962cc41c27b service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.128803 May 25 16:19:51.318595 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 190/388] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:51.331789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f8e2bd7d-b468-4bb3-af60-f15ef81cffef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:51.332902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f8e2bd7d-b468-4bb3-af60-f15ef81cffef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:51.342947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f8e2bd7d-b468-4bb3-af60-f15ef81cffef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:51.343837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f8e2bd7d-b468-4bb3-af60-f15ef81cffef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:51.344972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f8e2bd7d-b468-4bb3-af60-f15ef81cffef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:51.346526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f8e2bd7d-b468-4bb3-af60-f15ef81cffef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:51.607981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6427f555-9c68-4472-9cb2-f0d14a717a9f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:51.612147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f8e2bd7d-b468-4bb3-af60-f15ef81cffef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:51.620379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f8e2bd7d-b468-4bb3-af60-f15ef81cffef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.295064 May 25 16:19:51.620954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 191/389] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:51 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:51.624208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6427f555-9c68-4472-9cb2-f0d14a717a9f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.379287 May 25 16:19:51.627311 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-718a0f36-94ca-48ba-8593-8fc8dbcba8b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:51.627967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 199/390] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:51 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:51.638325 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-718a0f36-94ca-48ba-8593-8fc8dbcba8b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:51.645073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-718a0f36-94ca-48ba-8593-8fc8dbcba8b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:51.645521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-718a0f36-94ca-48ba-8593-8fc8dbcba8b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:51.646365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-718a0f36-94ca-48ba-8593-8fc8dbcba8b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:51.646795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-718a0f36-94ca-48ba-8593-8fc8dbcba8b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:51.669191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-eef7a00a-e1fe-44e6-b695-06d86450cc96 req-68445078-458b-4568-9406-f0e0264b4309 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4", "name": "network-vif-plugged", "status": "completed", "tag": "4d5ed500-2623-48b8-bc81-a72eb4ffeb2d"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:51.669582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:51.679044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-eef7a00a-e1fe-44e6-b695-06d86450cc96 req-68445078-458b-4568-9406-f0e0264b4309 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:51.679849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-eef7a00a-e1fe-44e6-b695-06d86450cc96 req-68445078-458b-4568-9406-f0e0264b4309 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:51.699512 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-eef7a00a-e1fe-44e6-b695-06d86450cc96 req-68445078-458b-4568-9406-f0e0264b4309 service nova] Creating event network-vif-plugged:4d5ed500-2623-48b8-bc81-a72eb4ffeb2d for instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 on hv-cinder-84327 May 25 16:19:51.706908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-eef7a00a-e1fe-44e6-b695-06d86450cc96 req-68445078-458b-4568-9406-f0e0264b4309 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041563 May 25 16:19:51.707837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 200/391] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:19:51.800564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-718a0f36-94ca-48ba-8593-8fc8dbcba8b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:51.809444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-718a0f36-94ca-48ba-8593-8fc8dbcba8b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.186798 May 25 16:19:51.810285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 192/392] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:51 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:52.191688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9c0e46c7-8589-4185-8c81-0d819c6f53e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:52.192497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9c0e46c7-8589-4185-8c81-0d819c6f53e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:52.199298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c0e46c7-8589-4185-8c81-0d819c6f53e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:52.199732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c0e46c7-8589-4185-8c81-0d819c6f53e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:52.200515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c0e46c7-8589-4185-8c81-0d819c6f53e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:52.200945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c0e46c7-8589-4185-8c81-0d819c6f53e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:52.258622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:52.259451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:52.262647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:52.268335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:52.268956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:52.269926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:52.270769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:52.393457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9c0e46c7-8589-4185-8c81-0d819c6f53e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:52.405008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9c0e46c7-8589-4185-8c81-0d819c6f53e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.217426 May 25 16:19:52.468978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 201/393] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:52 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:52.549712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:52.564708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:52.566410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:52.567754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-60447399-6334-449d-b536-83c4bf1bd789 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1740 microversion: 2.20 time: 0.313026 May 25 16:19:52.568844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 193/394] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1740 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:52.638304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7325bf5a-d5c1-457f-a3fb-b16a4f4b7d5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:52.639220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7325bf5a-d5c1-457f-a3fb-b16a4f4b7d5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:52.644990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e3e22f57-99b5-417c-875d-af9ba284a593 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:52.645567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e3e22f57-99b5-417c-875d-af9ba284a593 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:52.647305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7325bf5a-d5c1-457f-a3fb-b16a4f4b7d5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:52.647947 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7325bf5a-d5c1-457f-a3fb-b16a4f4b7d5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:52.652358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3e22f57-99b5-417c-875d-af9ba284a593 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:52.652793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3e22f57-99b5-417c-875d-af9ba284a593 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:52.653714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3e22f57-99b5-417c-875d-af9ba284a593 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:52.654418 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3e22f57-99b5-417c-875d-af9ba284a593 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:52.657262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7325bf5a-d5c1-457f-a3fb-b16a4f4b7d5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:52.658603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7325bf5a-d5c1-457f-a3fb-b16a4f4b7d5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:52.983609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e3e22f57-99b5-417c-875d-af9ba284a593 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:52.997215 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e3e22f57-99b5-417c-875d-af9ba284a593 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.356279 May 25 16:19:53.006958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 194/395] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:52 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:53.013491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7325bf5a-d5c1-457f-a3fb-b16a4f4b7d5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:53.022694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7325bf5a-d5c1-457f-a3fb-b16a4f4b7d5e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.388410 May 25 16:19:53.025152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1704b095-6cd7-4ca8-b32b-43d211b679d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:53.025767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1704b095-6cd7-4ca8-b32b-43d211b679d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:53.027107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 202/396] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:52 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:53.046190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1704b095-6cd7-4ca8-b32b-43d211b679d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:53.047053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1704b095-6cd7-4ca8-b32b-43d211b679d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:53.048064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1704b095-6cd7-4ca8-b32b-43d211b679d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:53.048658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1704b095-6cd7-4ca8-b32b-43d211b679d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:53.232006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1704b095-6cd7-4ca8-b32b-43d211b679d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:53.239443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1704b095-6cd7-4ca8-b32b-43d211b679d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.231099 May 25 16:19:53.240012 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 195/397] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:53 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:53.453924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-39334a8e-1589-4fc5-9b57-94d4144abdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:53.454495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-39334a8e-1589-4fc5-9b57-94d4144abdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:53.461290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39334a8e-1589-4fc5-9b57-94d4144abdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:53.461728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39334a8e-1589-4fc5-9b57-94d4144abdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:53.462489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39334a8e-1589-4fc5-9b57-94d4144abdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:53.463515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39334a8e-1589-4fc5-9b57-94d4144abdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:53.592070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:53.593320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:53.607336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-39334a8e-1589-4fc5-9b57-94d4144abdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:53.613499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:53.614266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:53.615211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:53.616907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-39334a8e-1589-4fc5-9b57-94d4144abdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.166797 May 25 16:19:53.617496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 203/398] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:53 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:53.617720 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:53.777901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:53.788284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:53.789886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:53.791483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0ff4cd35-badd-4a4d-9aa9-4868a270d74b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1740 microversion: 2.20 time: 0.209184 May 25 16:19:53.792272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 196/399] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:53 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1740 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:54.025303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-25a78577-7977-4fcd-aa3a-d18537bc9f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:54.026988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-25a78577-7977-4fcd-aa3a-d18537bc9f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:54.031289 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:54.044112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c653e5ce-d762-4d7f-b4d1-df9c8da7cd5d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:54.045038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c653e5ce-d762-4d7f-b4d1-df9c8da7cd5d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:54.049521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25a78577-7977-4fcd-aa3a-d18537bc9f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.051702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25a78577-7977-4fcd-aa3a-d18537bc9f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.052284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c653e5ce-d762-4d7f-b4d1-df9c8da7cd5d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.052802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25a78577-7977-4fcd-aa3a-d18537bc9f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.053156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c653e5ce-d762-4d7f-b4d1-df9c8da7cd5d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.053719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25a78577-7977-4fcd-aa3a-d18537bc9f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.054295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c653e5ce-d762-4d7f-b4d1-df9c8da7cd5d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.054918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c653e5ce-d762-4d7f-b4d1-df9c8da7cd5d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.516442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-25a78577-7977-4fcd-aa3a-d18537bc9f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:54.526171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-25a78577-7977-4fcd-aa3a-d18537bc9f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.506039 May 25 16:19:54.526598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c653e5ce-d762-4d7f-b4d1-df9c8da7cd5d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:54.531542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 204/400] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:54 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:54.537092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-61adf64f-7486-4d17-9dd7-52b2f0eb59ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:54.537776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-61adf64f-7486-4d17-9dd7-52b2f0eb59ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:54.538410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c653e5ce-d762-4d7f-b4d1-df9c8da7cd5d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.498100 May 25 16:19:54.542185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 197/401] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:54 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:54.544913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-61adf64f-7486-4d17-9dd7-52b2f0eb59ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.545273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-61adf64f-7486-4d17-9dd7-52b2f0eb59ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.546114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-61adf64f-7486-4d17-9dd7-52b2f0eb59ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.546612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-61adf64f-7486-4d17-9dd7-52b2f0eb59ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.634759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5340336e-7c15-4c52-a6ae-b5493afd1681 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:54.635937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5340336e-7c15-4c52-a6ae-b5493afd1681 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:54.642684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5340336e-7c15-4c52-a6ae-b5493afd1681 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.643303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5340336e-7c15-4c52-a6ae-b5493afd1681 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.644154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5340336e-7c15-4c52-a6ae-b5493afd1681 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.644661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5340336e-7c15-4c52-a6ae-b5493afd1681 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.902827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5340336e-7c15-4c52-a6ae-b5493afd1681 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:54.920259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5340336e-7c15-4c52-a6ae-b5493afd1681 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1357 microversion: 2.1 time: 0.289575 May 25 16:19:54.920848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 198/402] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:54 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1357 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:54.925462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:54.926030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:54.939956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.940811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.941869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:54.942567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:54.976094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-61adf64f-7486-4d17-9dd7-52b2f0eb59ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:54.987837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-61adf64f-7486-4d17-9dd7-52b2f0eb59ba tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.454806 May 25 16:19:54.988416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 205/403] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:54 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:55.157084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:55.169002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:55.170736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:55.172199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ae42670c-240a-4960-92af-ab20134fc26a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1740 microversion: 2.20 time: 0.250423 May 25 16:19:55.172871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 199/404] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1740 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:55.424651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-98744a76-d089-4afb-acaa-9fa3c433c522 req-6bd1bed0-9d00-4f63-bc4a-6fc07eece274 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "667322c4-d9ca-4320-903f-112d84e5f462", "name": "network-vif-plugged", "status": "completed", "tag": "0906f7f3-63c4-42d4-9006-1ed37e306e92"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:55.433481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-98744a76-d089-4afb-acaa-9fa3c433c522 req-6bd1bed0-9d00-4f63-bc4a-6fc07eece274 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:55.434254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-98744a76-d089-4afb-acaa-9fa3c433c522 req-6bd1bed0-9d00-4f63-bc4a-6fc07eece274 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:55.450913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-98744a76-d089-4afb-acaa-9fa3c433c522 req-6bd1bed0-9d00-4f63-bc4a-6fc07eece274 service nova] Creating event network-vif-plugged:0906f7f3-63c4-42d4-9006-1ed37e306e92 for instance 667322c4-d9ca-4320-903f-112d84e5f462 on hv-cinder-84327 May 25 16:19:55.456887 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-98744a76-d089-4afb-acaa-9fa3c433c522 req-6bd1bed0-9d00-4f63-bc4a-6fc07eece274 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035681 May 25 16:19:55.457979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 206/405] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:55.544213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-81e5444f-12be-4767-b6e1-726a87c44b17 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:55.544911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-81e5444f-12be-4767-b6e1-726a87c44b17 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:55.558695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-800a061b-2dfc-439c-8cca-cf7611c7f228 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:55.559872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-800a061b-2dfc-439c-8cca-cf7611c7f228 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:55.565678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-81e5444f-12be-4767-b6e1-726a87c44b17 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:55.566355 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-81e5444f-12be-4767-b6e1-726a87c44b17 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:55.567491 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-81e5444f-12be-4767-b6e1-726a87c44b17 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:55.569332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-81e5444f-12be-4767-b6e1-726a87c44b17 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:55.570596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-800a061b-2dfc-439c-8cca-cf7611c7f228 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:55.573085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-800a061b-2dfc-439c-8cca-cf7611c7f228 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:55.578033 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-800a061b-2dfc-439c-8cca-cf7611c7f228 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:55.578876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-800a061b-2dfc-439c-8cca-cf7611c7f228 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:55.907962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-81e5444f-12be-4767-b6e1-726a87c44b17 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:55.915713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-81e5444f-12be-4767-b6e1-726a87c44b17 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.376265 May 25 16:19:55.916639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 200/406] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:55 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:55.916836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-800a061b-2dfc-439c-8cca-cf7611c7f228 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:55.934516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-800a061b-2dfc-439c-8cca-cf7611c7f228 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.379825 May 25 16:19:55.935795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 207/407] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:55 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:55.937839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a87595e9-dc5a-422a-9845-0b23ea06844c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:55.938581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a87595e9-dc5a-422a-9845-0b23ea06844c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:55.944924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a87595e9-dc5a-422a-9845-0b23ea06844c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:55.946812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a87595e9-dc5a-422a-9845-0b23ea06844c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:55.947820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a87595e9-dc5a-422a-9845-0b23ea06844c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:55.948409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a87595e9-dc5a-422a-9845-0b23ea06844c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.006309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5ab25fbf-dfe0-41f4-a583-f0cd174ed7e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:56.007360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5ab25fbf-dfe0-41f4-a583-f0cd174ed7e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:56.015511 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ab25fbf-dfe0-41f4-a583-f0cd174ed7e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.016314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ab25fbf-dfe0-41f4-a583-f0cd174ed7e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.017444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ab25fbf-dfe0-41f4-a583-f0cd174ed7e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.018096 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ab25fbf-dfe0-41f4-a583-f0cd174ed7e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.174331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a87595e9-dc5a-422a-9845-0b23ea06844c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:56.183369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a87595e9-dc5a-422a-9845-0b23ea06844c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1547 microversion: 2.1 time: 0.250391 May 25 16:19:56.187586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 201/408] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:55 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1547 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:56.200767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:56.201682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:56.208756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.208756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.209627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.210161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.235214 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5ab25fbf-dfe0-41f4-a583-f0cd174ed7e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:56.243326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5ab25fbf-dfe0-41f4-a583-f0cd174ed7e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.241720 May 25 16:19:56.244404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 208/409] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:56 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:56.249218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a47c56c5-8957-4de9-b947-c779e592df5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:56.249824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a47c56c5-8957-4de9-b947-c779e592df5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:56.256093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a47c56c5-8957-4de9-b947-c779e592df5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.256542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a47c56c5-8957-4de9-b947-c779e592df5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.257531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a47c56c5-8957-4de9-b947-c779e592df5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.257969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a47c56c5-8957-4de9-b947-c779e592df5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.404635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:56.418173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:56.419667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:56.420910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-33997c59-5f5f-423b-b317-aa16c6f14802 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1740 microversion: 2.20 time: 0.232207 May 25 16:19:56.421728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 202/410] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:56 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1740 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:56.447995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a47c56c5-8957-4de9-b947-c779e592df5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:56.456349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a47c56c5-8957-4de9-b947-c779e592df5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1547 microversion: 2.1 time: 0.210692 May 25 16:19:56.456949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 209/411] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:56 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1547 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:56.473301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bea6167a-1897-4b6b-8226-7e1a6742c9f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:56.477040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bea6167a-1897-4b6b-8226-7e1a6742c9f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:56.477833 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bea6167a-1897-4b6b-8226-7e1a6742c9f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Searching by: {'deleted': False, 'project_id': '711b13e4ea3e4269a0082811f6708d87'} {{(pid=117795) get_all /opt/stack/nova/nova/compute/api.py:2962}} May 25 16:19:56.485248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bea6167a-1897-4b6b-8226-7e1a6742c9f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.485701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bea6167a-1897-4b6b-8226-7e1a6742c9f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.486535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bea6167a-1897-4b6b-8226-7e1a6742c9f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.486975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bea6167a-1897-4b6b-8226-7e1a6742c9f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.607965 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.multi_cell_list [None req-bea6167a-1897-4b6b-8226-7e1a6742c9f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=117795) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} May 25 16:19:56.719703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bea6167a-1897-4b6b-8226-7e1a6742c9f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.250216 May 25 16:19:56.720482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 203/412] 10.222.0.45 () {58 vars in 1124 bytes} [Wed May 25 16:19:56 2022] GET /compute/v2.1/servers => generated 342 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:56.730756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b207d17d-245e-48e6-af68-813c38dcb3b5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:56.731650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b207d17d-245e-48e6-af68-813c38dcb3b5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:56.739993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b207d17d-245e-48e6-af68-813c38dcb3b5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.740468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b207d17d-245e-48e6-af68-813c38dcb3b5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.741312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b207d17d-245e-48e6-af68-813c38dcb3b5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.741821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b207d17d-245e-48e6-af68-813c38dcb3b5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.880209 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b207d17d-245e-48e6-af68-813c38dcb3b5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:56.888172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b207d17d-245e-48e6-af68-813c38dcb3b5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1547 microversion: 2.1 time: 0.161310 May 25 16:19:56.899720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 210/413] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:56 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1547 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:56.932944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e46b1609-0703-4640-ab5a-0fc87d98e8ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:56.934184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e46b1609-0703-4640-ab5a-0fc87d98e8ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:56.940626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e46b1609-0703-4640-ab5a-0fc87d98e8ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.941269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e46b1609-0703-4640-ab5a-0fc87d98e8ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.942243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e46b1609-0703-4640-ab5a-0fc87d98e8ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.942828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e46b1609-0703-4640-ab5a-0fc87d98e8ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.952071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1e53f25a-4cfa-47ff-8666-adf1252c5569 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:56.952879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1e53f25a-4cfa-47ff-8666-adf1252c5569 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:56.962385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e53f25a-4cfa-47ff-8666-adf1252c5569 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.963197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e53f25a-4cfa-47ff-8666-adf1252c5569 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:56.968690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e53f25a-4cfa-47ff-8666-adf1252c5569 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:56.969895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e53f25a-4cfa-47ff-8666-adf1252c5569 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:57.269853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e46b1609-0703-4640-ab5a-0fc87d98e8ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:57.279130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e46b1609-0703-4640-ab5a-0fc87d98e8ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1366 microversion: 2.1 time: 0.350173 May 25 16:19:57.291717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 204/414] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:56 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1366 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:57.296306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1e53f25a-4cfa-47ff-8666-adf1252c5569 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:57.297309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7bd653ec-b46d-484c-ac02-a32efabbdf7c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:57.297934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7bd653ec-b46d-484c-ac02-a32efabbdf7c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:57.304400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7bd653ec-b46d-484c-ac02-a32efabbdf7c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:57.304854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7bd653ec-b46d-484c-ac02-a32efabbdf7c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:57.306847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7bd653ec-b46d-484c-ac02-a32efabbdf7c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:57.308268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1e53f25a-4cfa-47ff-8666-adf1252c5569 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.359769 May 25 16:19:57.311227 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7bd653ec-b46d-484c-ac02-a32efabbdf7c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:57.311765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 211/415] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:56 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:57.440823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:57.441714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:57.458024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:57.458552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:57.459369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:57.459804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:57.471930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7bd653ec-b46d-484c-ac02-a32efabbdf7c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:57.482686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7bd653ec-b46d-484c-ac02-a32efabbdf7c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.189651 May 25 16:19:57.484290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 205/416] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:57 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:57.648417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-1940ac50-6175-4716-8568-dea5d6fa2ae6 req-4301cfee-d9fe-437c-8746-592c933cb7da service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "516581ca-a13c-4a28-92da-988be630b29c", "name": "network-vif-plugged", "status": "completed", "tag": "798e662c-998a-4c12-afb9-3d2641d4ff07"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:57.661830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:57.664864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-1940ac50-6175-4716-8568-dea5d6fa2ae6 req-4301cfee-d9fe-437c-8746-592c933cb7da service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:57.666957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-1940ac50-6175-4716-8568-dea5d6fa2ae6 req-4301cfee-d9fe-437c-8746-592c933cb7da service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:57.677292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:57.678889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:57.679969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-21f53b32-5e7a-4dea-b67c-180084610ab2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1740 microversion: 2.20 time: 0.244478 May 25 16:19:57.681511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-1940ac50-6175-4716-8568-dea5d6fa2ae6 req-4301cfee-d9fe-437c-8746-592c933cb7da service nova] Creating event network-vif-plugged:798e662c-998a-4c12-afb9-3d2641d4ff07 for instance 516581ca-a13c-4a28-92da-988be630b29c on hv-cinder-84327 May 25 16:19:57.687240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-1940ac50-6175-4716-8568-dea5d6fa2ae6 req-4301cfee-d9fe-437c-8746-592c933cb7da service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042887 May 25 16:19:57.688481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 206/417] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:57.695420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 212/418] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1740 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:19:58.300854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-01b0b77b-1acf-4449-8ab4-11e15a3162e0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:58.302407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-01b0b77b-1acf-4449-8ab4-11e15a3162e0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:58.326947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01b0b77b-1acf-4449-8ab4-11e15a3162e0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.328208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01b0b77b-1acf-4449-8ab4-11e15a3162e0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.328339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dbc3834f-81ee-414c-950a-a86940d2deed tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:58.328924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dbc3834f-81ee-414c-950a-a86940d2deed tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:58.330967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01b0b77b-1acf-4449-8ab4-11e15a3162e0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.333817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01b0b77b-1acf-4449-8ab4-11e15a3162e0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.342220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dbc3834f-81ee-414c-950a-a86940d2deed tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.344373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dbc3834f-81ee-414c-950a-a86940d2deed tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.344373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dbc3834f-81ee-414c-950a-a86940d2deed tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.344373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dbc3834f-81ee-414c-950a-a86940d2deed tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.681450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dbc3834f-81ee-414c-950a-a86940d2deed tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:58.684788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:58.689623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-01b0b77b-1acf-4449-8ab4-11e15a3162e0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:58.693103 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dbc3834f-81ee-414c-950a-a86940d2deed tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.370740 May 25 16:19:58.698035 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-01b0b77b-1acf-4449-8ab4-11e15a3162e0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1556 microversion: 2.1 time: 0.403739 May 25 16:19:58.698605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 207/419] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:58 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1556 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:19:58.702911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 213/420] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:58 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:58.704445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e6db982d-579d-40fc-82d6-e09da8903805 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:58.705253 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e6db982d-579d-40fc-82d6-e09da8903805 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:58.712628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6db982d-579d-40fc-82d6-e09da8903805 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.713078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6db982d-579d-40fc-82d6-e09da8903805 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.713927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6db982d-579d-40fc-82d6-e09da8903805 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.714547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6db982d-579d-40fc-82d6-e09da8903805 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.719796 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4c79bf97-70a0-495f-b93d-b48a8ead57c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8b19afac-6e47-40fb-a1df-82a200122a51"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:58.722132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4c79bf97-70a0-495f-b93d-b48a8ead57c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:58.730045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c79bf97-70a0-495f-b93d-b48a8ead57c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.730689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c79bf97-70a0-495f-b93d-b48a8ead57c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.731692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c79bf97-70a0-495f-b93d-b48a8ead57c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.732277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c79bf97-70a0-495f-b93d-b48a8ead57c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.864410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e6db982d-579d-40fc-82d6-e09da8903805 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:58.873074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e6db982d-579d-40fc-82d6-e09da8903805 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.173450 May 25 16:19:58.873902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 208/421] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:58 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:19:58.883242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:58.887231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:58.893949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.894534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.895239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:58.895669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:58.911336 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:59.077153 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:59.089248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:59.091128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:19:59.092320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f5a6acf6-d730-43f1-9570-a4afb6883dc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1740 microversion: 2.20 time: 0.217237 May 25 16:19:59.092925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 209/422] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:19:58 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1740 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:19:59.099413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-485e5d08-1e0f-4545-a784-828b2c8e1699 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:59.100110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-485e5d08-1e0f-4545-a784-828b2c8e1699 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:59.106494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-485e5d08-1e0f-4545-a784-828b2c8e1699 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:59.106931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-485e5d08-1e0f-4545-a784-828b2c8e1699 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:59.107709 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-485e5d08-1e0f-4545-a784-828b2c8e1699 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:59.108289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-485e5d08-1e0f-4545-a784-828b2c8e1699 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:59.199444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-485e5d08-1e0f-4545-a784-828b2c8e1699 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.105371 May 25 16:19:59.200002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 210/423] 10.222.0.45 () {58 vars in 1274 bytes} [Wed May 25 16:19:59 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462/os-interface => generated 284 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:59.218561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4c79bf97-70a0-495f-b93d-b48a8ead57c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "POST /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.502767 May 25 16:19:59.219708 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 214/424] 10.222.0.45 () {60 vars in 1322 bytes} [Wed May 25 16:19:58 2022] POST /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments => generated 194 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:59.729319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a046b94f-f500-41c8-a1e6-7c16b109fbb2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:59.731897 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-f6ee77f7-1694-4d5a-8634-1c1cb79f9274 req-0d3c3964-407f-4db2-8135-42545e9af70d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "621e54b1-cdde-493f-9bad-1c1222a0731a", "name": "network-vif-plugged", "status": "completed", "tag": "9b36da50-ea60-4505-bbbd-48a647e0698b"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:19:59.735822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a046b94f-f500-41c8-a1e6-7c16b109fbb2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:59.747809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a046b94f-f500-41c8-a1e6-7c16b109fbb2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:59.748866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a046b94f-f500-41c8-a1e6-7c16b109fbb2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:59.750425 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a046b94f-f500-41c8-a1e6-7c16b109fbb2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:59.750864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a046b94f-f500-41c8-a1e6-7c16b109fbb2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:59.756723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-f6ee77f7-1694-4d5a-8634-1c1cb79f9274 req-0d3c3964-407f-4db2-8135-42545e9af70d service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:59.759630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-f6ee77f7-1694-4d5a-8634-1c1cb79f9274 req-0d3c3964-407f-4db2-8135-42545e9af70d service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:59.785860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-f6ee77f7-1694-4d5a-8634-1c1cb79f9274 req-0d3c3964-407f-4db2-8135-42545e9af70d service nova] Creating event network-vif-plugged:9b36da50-ea60-4505-bbbd-48a647e0698b for instance 621e54b1-cdde-493f-9bad-1c1222a0731a on hv-cinder-84327 May 25 16:19:59.795537 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-f6ee77f7-1694-4d5a-8634-1c1cb79f9274 req-0d3c3964-407f-4db2-8135-42545e9af70d service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072846 May 25 16:19:59.801433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 215/425] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:19:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:19:59.900184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ee5c6d5b-4b9f-4964-884f-d59b21ad4d45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:19:59.901299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ee5c6d5b-4b9f-4964-884f-d59b21ad4d45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:19:59.921994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee5c6d5b-4b9f-4964-884f-d59b21ad4d45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:59.924994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee5c6d5b-4b9f-4964-884f-d59b21ad4d45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:19:59.925884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee5c6d5b-4b9f-4964-884f-d59b21ad4d45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:19:59.930199 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee5c6d5b-4b9f-4964-884f-d59b21ad4d45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:00.113246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a046b94f-f500-41c8-a1e6-7c16b109fbb2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:00.123114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a046b94f-f500-41c8-a1e6-7c16b109fbb2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1319 microversion: 2.1 time: 0.405975 May 25 16:20:00.124490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 211/426] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:59 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1319 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:00.129883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:00.130736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:00.139274 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:00.139719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:00.140783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:00.141287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:00.318622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ee5c6d5b-4b9f-4964-884f-d59b21ad4d45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:00.326605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ee5c6d5b-4b9f-4964-884f-d59b21ad4d45 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.439495 May 25 16:20:00.337586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 216/427] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:19:59 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:00.460214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:00.469943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:00.476627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:00.478096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2cae80f7-5022-4f57-8940-835592b20155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1740 microversion: 2.20 time: 0.352214 May 25 16:20:00.487405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 212/428] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:00 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1740 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:00.635029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-6b819eb6-064f-48e8-91a9-fa1d564cd6d0 req-7afedeb5-27d5-46d3-a0d3-df9c898d5fb1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "667322c4-d9ca-4320-903f-112d84e5f462", "tag": "0906f7f3-63c4-42d4-9006-1ed37e306e92"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:00.644224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-6b819eb6-064f-48e8-91a9-fa1d564cd6d0 req-7afedeb5-27d5-46d3-a0d3-df9c898d5fb1 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:00.644728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-6b819eb6-064f-48e8-91a9-fa1d564cd6d0 req-7afedeb5-27d5-46d3-a0d3-df9c898d5fb1 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:00.657277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-6b819eb6-064f-48e8-91a9-fa1d564cd6d0 req-7afedeb5-27d5-46d3-a0d3-df9c898d5fb1 service nova] Creating event network-changed:0906f7f3-63c4-42d4-9006-1ed37e306e92 for instance 667322c4-d9ca-4320-903f-112d84e5f462 on hv-cinder-84327 May 25 16:20:00.663600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-6b819eb6-064f-48e8-91a9-fa1d564cd6d0 req-7afedeb5-27d5-46d3-a0d3-df9c898d5fb1 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033024 May 25 16:20:00.664483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 217/429] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:20:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:20:01.140782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-53fcce25-85ab-43aa-9f00-a29855e83209 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:01.141420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-53fcce25-85ab-43aa-9f00-a29855e83209 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:01.148150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-53fcce25-85ab-43aa-9f00-a29855e83209 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:01.148604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-53fcce25-85ab-43aa-9f00-a29855e83209 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:01.149366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-53fcce25-85ab-43aa-9f00-a29855e83209 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:01.149806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-53fcce25-85ab-43aa-9f00-a29855e83209 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:01.278455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-53fcce25-85ab-43aa-9f00-a29855e83209 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:01.286241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-53fcce25-85ab-43aa-9f00-a29855e83209 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1506 microversion: 2.1 time: 0.149321 May 25 16:20:01.286804 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 213/430] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:20:01 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1506 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:01.344097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2faec011-7294-4966-a123-220b40ad9a3a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:01.344785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2faec011-7294-4966-a123-220b40ad9a3a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:01.352050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2faec011-7294-4966-a123-220b40ad9a3a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:01.352623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2faec011-7294-4966-a123-220b40ad9a3a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:01.353664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2faec011-7294-4966-a123-220b40ad9a3a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:01.354401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2faec011-7294-4966-a123-220b40ad9a3a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:01.496251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:01.496856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:01.503554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:01.503991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:01.505288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:01.505871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:01.529412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2faec011-7294-4966-a123-220b40ad9a3a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:01.537833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2faec011-7294-4966-a123-220b40ad9a3a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.197885 May 25 16:20:01.538747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 218/431] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:20:01 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:01.648644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:01.657864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:01.659390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:01.660602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6d73a681-1b00-4ea1-9f70-836ecd482de1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1740 microversion: 2.20 time: 0.168343 May 25 16:20:01.661157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 214/432] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:01 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1740 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:20:02.555954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-759c649b-6e7b-4c91-ba87-a782598200d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:02.556509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-759c649b-6e7b-4c91-ba87-a782598200d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:02.563414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-759c649b-6e7b-4c91-ba87-a782598200d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:02.564302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-759c649b-6e7b-4c91-ba87-a782598200d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:02.565094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-759c649b-6e7b-4c91-ba87-a782598200d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:02.565532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-759c649b-6e7b-4c91-ba87-a782598200d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:02.679247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:02.680163 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:02.687522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:02.687977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:02.688838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:02.689279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:02.948386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:02.951314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-759c649b-6e7b-4c91-ba87-a782598200d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:02.958641 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:02.959662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-759c649b-6e7b-4c91-ba87-a782598200d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1372 microversion: 2.1 time: 0.407744 May 25 16:20:02.960181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:02.960662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 219/433] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:20:02 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1372 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:02.961721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-45282d7a-70e1-497e-8f2b-947940b9f524 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1722 microversion: 2.20 time: 0.286768 May 25 16:20:02.962256 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 215/434] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:02 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1722 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:03.977103 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-314e01e0-937a-4e34-8111-2487121f7f02 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:03.977639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-314e01e0-937a-4e34-8111-2487121f7f02 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:03.980505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:03.981094 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:03.984429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-314e01e0-937a-4e34-8111-2487121f7f02 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:03.984990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-314e01e0-937a-4e34-8111-2487121f7f02 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:03.985847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-314e01e0-937a-4e34-8111-2487121f7f02 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:03.986356 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-314e01e0-937a-4e34-8111-2487121f7f02 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:03.987878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:03.988331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:03.989301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:03.989725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:04.275656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-314e01e0-937a-4e34-8111-2487121f7f02 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:04.280633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:04.283368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-314e01e0-937a-4e34-8111-2487121f7f02 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1566 microversion: 2.1 time: 0.310725 May 25 16:20:04.283963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 220/435] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:20:03 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1566 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:04.289068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:04.290583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:04.292089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2d0186fd-0a3b-4ae0-aecd-b8e3bfeb12ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1722 microversion: 2.20 time: 0.315620 May 25 16:20:04.292615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 216/436] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:03 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1722 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:20:04.302851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9ad37e25-6bbc-4881-a3c9-24dc60757e91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:04.303375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9ad37e25-6bbc-4881-a3c9-24dc60757e91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:04.309777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ad37e25-6bbc-4881-a3c9-24dc60757e91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:04.310220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ad37e25-6bbc-4881-a3c9-24dc60757e91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:04.310999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ad37e25-6bbc-4881-a3c9-24dc60757e91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:04.311491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ad37e25-6bbc-4881-a3c9-24dc60757e91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:04.405476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9ad37e25-6bbc-4881-a3c9-24dc60757e91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.106453 May 25 16:20:04.406234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 221/437] 10.222.0.45 () {58 vars in 1274 bytes} [Wed May 25 16:20:04 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-interface => generated 285 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:20:05.310192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:05.310811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:05.318059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:05.318352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:05.319309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:05.319581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:05.434582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-4f41b1f7-e3aa-4af4-a996-567425538726 req-807a6285-92e0-438e-828e-3b4e506f3d8a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4", "name": "network-vif-unplugged", "status": "completed", "tag": "4d5ed500-2623-48b8-bc81-a72eb4ffeb2d"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:05.444373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-4f41b1f7-e3aa-4af4-a996-567425538726 req-807a6285-92e0-438e-828e-3b4e506f3d8a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:05.444815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-4f41b1f7-e3aa-4af4-a996-567425538726 req-807a6285-92e0-438e-828e-3b4e506f3d8a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:05.458625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-4f41b1f7-e3aa-4af4-a996-567425538726 req-807a6285-92e0-438e-828e-3b4e506f3d8a service nova] Creating event network-vif-unplugged:4d5ed500-2623-48b8-bc81-a72eb4ffeb2d for instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 on hv-cinder-84327 May 25 16:20:05.465808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-4f41b1f7-e3aa-4af4-a996-567425538726 req-807a6285-92e0-438e-828e-3b4e506f3d8a service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035237 May 25 16:20:05.467065 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 222/438] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:20:05 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:20:05.484071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:05.493714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:05.495338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:05.496511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6b1165dc-c4e9-4b42-b3aa-27bc68dbd373 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1722 microversion: 2.20 time: 0.190347 May 25 16:20:05.497360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 217/439] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:05 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1722 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:05.643710 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-b58f4bfc-15a3-4b2e-9c91-bcb94d8bda03 req-028b930c-791d-4cba-a7e3-6b28da158e6a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "621e54b1-cdde-493f-9bad-1c1222a0731a", "tag": "9b36da50-ea60-4505-bbbd-48a647e0698b"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:05.652163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-b58f4bfc-15a3-4b2e-9c91-bcb94d8bda03 req-028b930c-791d-4cba-a7e3-6b28da158e6a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:05.652609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-b58f4bfc-15a3-4b2e-9c91-bcb94d8bda03 req-028b930c-791d-4cba-a7e3-6b28da158e6a service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:05.664560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-b58f4bfc-15a3-4b2e-9c91-bcb94d8bda03 req-028b930c-791d-4cba-a7e3-6b28da158e6a service nova] Creating event network-changed:9b36da50-ea60-4505-bbbd-48a647e0698b for instance 621e54b1-cdde-493f-9bad-1c1222a0731a on hv-cinder-84327 May 25 16:20:05.670636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-b58f4bfc-15a3-4b2e-9c91-bcb94d8bda03 req-028b930c-791d-4cba-a7e3-6b28da158e6a service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030802 May 25 16:20:05.671875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 223/440] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:20:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:20:06.514315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:06.514951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:06.520869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:06.523381 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:06.524147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:06.525163 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:06.525866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:06.665365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:06.675578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:06.677109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:06.678291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1423ac4f-26b3-48b2-a6d4-72fd3d2c73ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1722 microversion: 2.20 time: 0.168120 May 25 16:20:06.679062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 218/441] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:06 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1722 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:07.698540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:07.699322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:07.706021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:07.706464 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:07.707249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:07.707692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:07.850219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:07.858230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:07.859832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:07.860984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1e4f79b9-e927-4d48-93ae-a1e9ecc54677 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1722 microversion: 2.20 time: 0.167346 May 25 16:20:07.861539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 224/442] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:07 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1722 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:08.884164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:08.884878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:08.891565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:08.892016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:08.892779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:08.893213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:08.968275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d72ad9d6-0499-4acb-883e-55a2243d8894 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:08.968794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d72ad9d6-0499-4acb-883e-55a2243d8894 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:08.975736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d72ad9d6-0499-4acb-883e-55a2243d8894 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:08.976371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d72ad9d6-0499-4acb-883e-55a2243d8894 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:08.977101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d72ad9d6-0499-4acb-883e-55a2243d8894 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:08.977700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d72ad9d6-0499-4acb-883e-55a2243d8894 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:09.032727 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:09.041198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:09.042692 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:09.043877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e3fa7c0e-eb4d-48e3-9d9c-45cadd406bd9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1725 microversion: 2.20 time: 0.164595 May 25 16:20:09.044469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 219/443] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:08 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1725 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:09.061232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-885389d8-1463-4524-85ed-2c0ad921c01b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "129ff07c-b9d7-4f99-9ec9-48c95b3d64ca"}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:09.062355 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-885389d8-1463-4524-85ed-2c0ad921c01b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:09.069570 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-885389d8-1463-4524-85ed-2c0ad921c01b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:09.070880 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-885389d8-1463-4524-85ed-2c0ad921c01b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:09.071902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-885389d8-1463-4524-85ed-2c0ad921c01b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:09.072524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-885389d8-1463-4524-85ed-2c0ad921c01b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:09.169569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d72ad9d6-0499-4acb-883e-55a2243d8894 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:09.187609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d72ad9d6-0499-4acb-883e-55a2243d8894 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1593 microversion: 2.1 time: 0.223110 May 25 16:20:09.193394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 225/444] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:20:08 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1593 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:09.479623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-885389d8-1463-4524-85ed-2c0ad921c01b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "POST /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.422185 May 25 16:20:09.485231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 220/445] 10.222.0.45 () {62 vars in 1363 bytes} [Wed May 25 16:20:09 2022] POST /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 188 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:20:09.605214 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c552056b-68a0-4a35-b524-39123671df32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:09.605868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c552056b-68a0-4a35-b524-39123671df32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:09.612024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c552056b-68a0-4a35-b524-39123671df32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:09.612470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c552056b-68a0-4a35-b524-39123671df32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:09.613214 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c552056b-68a0-4a35-b524-39123671df32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:09.613726 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c552056b-68a0-4a35-b524-39123671df32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:09.707623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c552056b-68a0-4a35-b524-39123671df32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "POST /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/action" status: 202 len: 0 microversion: 2.20 time: 0.106342 May 25 16:20:09.711652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 226/446] 10.222.0.45 () {62 vars in 1318 bytes} [Wed May 25 16:20:09 2022] POST /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/action => generated 0 bytes in 108 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 25 16:20:09.721160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:09.721805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:09.741154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:09.741611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:09.742386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:09.742869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:09.910076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:09.920245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:09.922181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:09.923467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-07105279-b41e-4e03-b12e-086513b72d21 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1751 microversion: 2.20 time: 0.206229 May 25 16:20:09.924062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 221/447] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:09 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1751 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:10.946083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:10.947425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:10.958784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:10.959959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:10.961021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:10.961673 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:11.141053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:11.150092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:11.151748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:11.152911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3c0c3879-1ad2-4543-82f5-877f78d927f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1753 microversion: 2.20 time: 0.214252 May 25 16:20:11.164948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 227/448] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:10 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1753 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:11.760490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-ef24e3ee-3930-4f36-b842-d47fba493715 req-ff23c722-24a9-4f09-80a0-bb6189def932 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d86354dc-f3d4-409d-b331-9f708947be2a", "tag": "e2c0938e-f9fb-4796-87fa-103cbcfd2536"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:11.797004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-ef24e3ee-3930-4f36-b842-d47fba493715 req-ff23c722-24a9-4f09-80a0-bb6189def932 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:11.800314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-ef24e3ee-3930-4f36-b842-d47fba493715 req-ff23c722-24a9-4f09-80a0-bb6189def932 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:11.818284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-ef24e3ee-3930-4f36-b842-d47fba493715 req-ff23c722-24a9-4f09-80a0-bb6189def932 service nova] Creating event network-changed:e2c0938e-f9fb-4796-87fa-103cbcfd2536 for instance d86354dc-f3d4-409d-b331-9f708947be2a on hv-cinder-84327 May 25 16:20:11.826486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-ef24e3ee-3930-4f36-b842-d47fba493715 req-ff23c722-24a9-4f09-80a0-bb6189def932 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068276 May 25 16:20:11.827803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 222/449] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:20:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:20:12.175756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:12.177227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:12.185195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:12.185919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:12.186709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:12.187141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:12.380844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:12.390968 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:12.392887 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:12.395764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-026659a9-b8e0-4dbc-9e13-2e1088510995 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.225502 May 25 16:20:12.396896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 228/450] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:12 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:13.173387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-40c9b7c7-c685-4b74-93d1-d6175b7064ba tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1161984986"}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:13.174007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-40c9b7c7-c685-4b74-93d1-d6175b7064ba tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:13.181477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40c9b7c7-c685-4b74-93d1-d6175b7064ba tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:13.181924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40c9b7c7-c685-4b74-93d1-d6175b7064ba tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:13.182688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40c9b7c7-c685-4b74-93d1-d6175b7064ba tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:13.183193 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40c9b7c7-c685-4b74-93d1-d6175b7064ba tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:13.392833 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.network.security_group_api [None req-40c9b7c7-c685-4b74-93d1-d6175b7064ba tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Adding security group f5a3b2c8-8714-4d7d-b283-b8771f400e61 to port e2c0938e-f9fb-4796-87fa-103cbcfd2536 May 25 16:20:13.414000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:13.414878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:13.422882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:13.423379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:13.424186 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:13.424608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:13.671088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:13.680986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:13.682470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:13.683853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6ed5cd8c-c65d-4b8b-9e08-e1e9ea88d4e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.273818 May 25 16:20:13.684594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 229/451] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:13 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:13.894950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-40c9b7c7-c685-4b74-93d1-d6175b7064ba tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "POST /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/action" status: 202 len: 0 microversion: 2.1 time: 0.725715 May 25 16:20:13.896086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 223/452] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:20:13 2022] POST /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/action => generated 0 bytes in 728 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:20:13.911182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bc254f70-a318-4123-a2c6-a144fb04d94d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:13.913693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-92343599-f856-4a99-b30b-e5d671457d67 req-0716223e-c73d-4e24-bd99-10688a274edb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d86354dc-f3d4-409d-b331-9f708947be2a", "tag": "e2c0938e-f9fb-4796-87fa-103cbcfd2536"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:13.919138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bc254f70-a318-4123-a2c6-a144fb04d94d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:13.929733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc254f70-a318-4123-a2c6-a144fb04d94d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:13.930175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc254f70-a318-4123-a2c6-a144fb04d94d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:13.930957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc254f70-a318-4123-a2c6-a144fb04d94d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:13.931415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc254f70-a318-4123-a2c6-a144fb04d94d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:13.936505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-92343599-f856-4a99-b30b-e5d671457d67 req-0716223e-c73d-4e24-bd99-10688a274edb service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:13.937121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-92343599-f856-4a99-b30b-e5d671457d67 req-0716223e-c73d-4e24-bd99-10688a274edb service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:13.968041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-92343599-f856-4a99-b30b-e5d671457d67 req-0716223e-c73d-4e24-bd99-10688a274edb service nova] Creating event network-changed:e2c0938e-f9fb-4796-87fa-103cbcfd2536 for instance d86354dc-f3d4-409d-b331-9f708947be2a on hv-cinder-84327 May 25 16:20:13.974060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-92343599-f856-4a99-b30b-e5d671457d67 req-0716223e-c73d-4e24-bd99-10688a274edb service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068714 May 25 16:20:13.979508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 224/453] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:20:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:20:14.209359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bc254f70-a318-4123-a2c6-a144fb04d94d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:14.218118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bc254f70-a318-4123-a2c6-a144fb04d94d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1759 microversion: 2.1 time: 0.316115 May 25 16:20:14.218720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 230/454] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:20:13 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1759 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:14.703475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:14.704108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:14.710852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:14.711320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:14.712083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:14.712602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:14.733501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:14.879298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:14.903305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:14.905455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:14.907532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-daaa47e0-84e6-439b-92f0-3bbcf793674b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.208773 May 25 16:20:14.908662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 225/455] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:14 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:15.928272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:15.929987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:15.937693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:15.938104 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:15.938871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:15.939338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:15.942329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:16.096679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:16.106146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:16.107679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:16.108935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f4efe3cc-7443-45e2-a75b-166f293444b9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.186332 May 25 16:20:16.112206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 231/456] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:15 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:17.128385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:17.129190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:17.142881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:17.143364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:17.144145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:17.144586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:17.295330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:17.304323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:17.305838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:17.307004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-aad54132-5521-4a8b-83d3-329607b77b65 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.183221 May 25 16:20:17.307890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 226/457] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:17 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:18.326168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:18.326823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:18.333306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:18.333748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:18.334475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:18.334899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:18.475371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:18.483872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:18.485363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:18.486516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-78ae7458-c4ae-46b4-b991-e31b86c73e13 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.164414 May 25 16:20:18.487077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 232/458] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:19.505803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:19.507217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:19.514068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:19.514524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:19.515327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:19.515750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:19.658077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:19.667855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:19.669354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:19.670557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a1cf4b43-48cf-45fa-96f4-32ef8071513d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.169357 May 25 16:20:19.681302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 227/459] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:19 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:20.688830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:20.689450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:20.696753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:20.697200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:20.698178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:20.698608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:20.850880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:20.860047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:20.861663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:20.862780 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c810a606-bf8c-4947-9f52-4e77688f4574 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.177920 May 25 16:20:20.863705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 233/460] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:20 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:21.882129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:21.882732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:21.890089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:21.890968 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:21.892062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:21.892559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:22.034348 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:22.044312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:22.046408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:22.048081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-29bd5ca0-5e88-4c0e-b764-0607dfde6dc4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.169866 May 25 16:20:22.049358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 228/461] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:21 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:23.072071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:23.072673 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:23.079774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:23.080236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:23.081028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:23.081538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:23.216415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:23.225821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:23.227682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:23.228949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a23079d9-44d8-4d1e-bea7-ac594458507c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.166458 May 25 16:20:23.229491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 234/462] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:23 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:24.104992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-499f4b60-4323-4522-ba7b-ee1f2c2b2fbd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e5dd7a55-9b8e-4463-83a8-1b7cc4540179", "device": "/dev/sdb"}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:24.107589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-499f4b60-4323-4522-ba7b-ee1f2c2b2fbd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:24.110930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=72 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:24.116627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-499f4b60-4323-4522-ba7b-ee1f2c2b2fbd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:24.117061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-499f4b60-4323-4522-ba7b-ee1f2c2b2fbd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:24.117821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-499f4b60-4323-4522-ba7b-ee1f2c2b2fbd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:24.118243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-499f4b60-4323-4522-ba7b-ee1f2c2b2fbd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:24.250342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:24.250968 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:24.255605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=64 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:24.263749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:24.264344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:24.265560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:24.266092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:24.412243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:24.421453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:24.422968 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:24.424201 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-27cc55cf-ab2b-4aa8-a1e2-4bcbe19a9d53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.180280 May 25 16:20:24.424734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 235/463] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:24 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:20:25.446454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:25.448038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:25.456444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:25.456897 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:25.457744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:25.458178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:25.598545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:25.612776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:25.614465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:25.615653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a336668c-0a18-4df6-95ff-bffdaf9fe238 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.174391 May 25 16:20:25.616480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 236/464] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:25 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:26.642168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:26.642869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:26.649700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:26.650254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:26.652389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:26.656141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:26.815996 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:26.824560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:26.826069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:26.827241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bc1e1a26-1ee9-4a82-881c-ea38b46ba816 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.189122 May 25 16:20:26.827806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 237/465] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:27.845403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:27.846106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:27.852935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:27.853373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:27.854277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:27.854695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:28.000768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:28.013076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:28.014866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:28.016073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5e65978d-9170-4ecd-97b4-f7a6c62e5cbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.174685 May 25 16:20:28.016636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 238/466] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:29.038534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:29.039673 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:29.051144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:29.051930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:29.053214 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:29.054071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:29.235336 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-499f4b60-4323-4522-ba7b-ee1f2c2b2fbd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "POST /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 5.134830 May 25 16:20:29.236269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 229/467] 10.222.0.45 () {60 vars in 1322 bytes} [Wed May 25 16:20:24 2022] POST /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c/os-volume_attachments => generated 194 bytes in 5137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:20:29.493763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:29.502966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:29.504523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:29.505733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-67b7b73e-bf9f-4591-ae1b-725b41e602cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.473857 May 25 16:20:29.506271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 239/468] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:29 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:30.529126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:30.530268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:30.539452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:30.539900 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:30.540672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:30.541436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:30.673479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:30.683333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:30.684837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:30.686002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5b743112-2b55-4b74-b5a1-5f55d2b9ca17 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.164073 May 25 16:20:30.686533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 230/469] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:30 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:31.708742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:31.710098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:31.718370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:31.718948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:31.720248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:31.720854 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:31.874407 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:31.884815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:31.886315 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:31.887503 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f5c10f00-3eac-4467-8d75-2b129dd1a7d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.185680 May 25 16:20:31.888092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 240/470] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:31 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:32.910056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:32.910871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:32.918104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:32.918482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:32.919263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:32.919693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:33.059444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:33.071506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:33.073339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:33.074674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-192d1227-bb2e-4f7e-97d3-0542e000a9f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.171279 May 25 16:20:33.075248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 231/471] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:32 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:34.094108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:34.094715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:34.102034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:34.102495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:34.103293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:34.103735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:34.239659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:34.248426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:34.249924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:34.251086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b29d1bf2-493e-4732-9a6e-5555e9ff232a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.160969 May 25 16:20:34.251678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 241/472] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:34 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:35.271886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:35.272760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:35.279302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:35.279693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:35.280436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:35.280862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:35.424126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:35.433543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:35.435041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:35.436222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-277bf668-e1f9-455f-8a33-372dea476cc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.169842 May 25 16:20:35.436790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 232/473] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:35 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:36.458170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:36.459068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:36.469902 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:36.470646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:36.471964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:36.472821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:36.615455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:36.624174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:36.625688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:36.626853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-df1fa413-3a2c-4b18-8d37-ceaf907bc495 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.175433 May 25 16:20:36.627553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 242/474] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:36 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:37.645689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:37.646266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:37.653359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:37.653812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:37.654571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:37.655009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:37.786938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:37.796296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:37.798050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:37.799245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1b54ba43-3ec0-474a-bba5-29a8623415d6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.157967 May 25 16:20:37.806896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 233/475] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:37 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:38.821307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:38.822784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:38.833741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:38.834483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:38.835790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:38.836509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:38.981186 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:38.990275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:38.991813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:38.992967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bd595dae-f540-4035-8d04-b9395ff8edcc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.178269 May 25 16:20:38.993509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 243/476] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:38 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:40.017050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:40.018040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:40.026610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:40.027063 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:40.028439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:40.029233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:40.169359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:40.185731 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:40.185731 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:40.185731 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cabba43d-ebcf-4666-8e8a-6bce5b650c36 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.170317 May 25 16:20:40.186111 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 234/477] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:40 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:41.199937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:41.200573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:41.207629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:41.208086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:41.208845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:41.209287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:41.349895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:41.363050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:41.364750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:41.365888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-da46f962-a4db-4e05-b61d-2670a41c83f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.169848 May 25 16:20:41.366448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 244/478] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:42.058334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-a11c9873-9670-4e75-8d31-7abe1a4f4283 req-e93e93b4-d60a-444b-b6a9-8b8e11803f72 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4", "name": "network-vif-plugged", "status": "completed", "tag": "4d5ed500-2623-48b8-bc81-a72eb4ffeb2d"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:42.067497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-a11c9873-9670-4e75-8d31-7abe1a4f4283 req-e93e93b4-d60a-444b-b6a9-8b8e11803f72 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:42.067988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-a11c9873-9670-4e75-8d31-7abe1a4f4283 req-e93e93b4-d60a-444b-b6a9-8b8e11803f72 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:42.080158 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-a11c9873-9670-4e75-8d31-7abe1a4f4283 req-e93e93b4-d60a-444b-b6a9-8b8e11803f72 service nova] Creating event network-vif-plugged:4d5ed500-2623-48b8-bc81-a72eb4ffeb2d for instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 on hv-cinder-84327 May 25 16:20:42.085965 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-a11c9873-9670-4e75-8d31-7abe1a4f4283 req-e93e93b4-d60a-444b-b6a9-8b8e11803f72 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031313 May 25 16:20:42.086627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 235/479] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:20:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:20:42.385245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:42.387200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:42.393674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:42.394129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:42.395020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:42.395497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:42.535174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:42.545385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:42.546899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:42.548089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-adc0c288-027a-4977-becd-631b71849375 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.167529 May 25 16:20:42.548671 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 245/480] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:43.575922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:43.577865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:43.587631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:43.588109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:43.588881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:43.589313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:43.727186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:43.736241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:43.737768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:43.739020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1598a50f-cde7-4c7a-8b0e-5f0f158b2164 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.174666 May 25 16:20:43.739597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 236/481] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:44.761818 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:44.762647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:44.773891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:44.774648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:44.775959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:44.776684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:44.916783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:44.928535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:44.930051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:44.931276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-66000d39-118d-4506-b3bd-98aa43a00e9b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.176064 May 25 16:20:44.931842 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 246/482] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:44 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:45.951494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:45.952208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:45.960116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:45.960555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:45.961323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:45.961767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:45.964540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=71,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:46.100385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:46.108994 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:46.110511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:46.111704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9a564c1f-ed7e-4e84-8875-70cd3a9eccc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.165239 May 25 16:20:46.112266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 237/483] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:45 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:47.130813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:47.131684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:47.140894 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:47.141497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:47.142511 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:47.143085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:47.146531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=96 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:47.288831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:47.301362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:47.302885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:47.304081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fe2dabfd-ac37-48ef-87ea-4897f900d943 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.177872 May 25 16:20:47.304615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 247/484] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:48.324341 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:48.325039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:48.333968 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:48.334576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:48.335918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:48.336507 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:48.482016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:48.490823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:48.492365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:48.493524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f3778d91-307e-4455-9d97-5c904124e7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.174893 May 25 16:20:48.494079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 238/485] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:49.515686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:49.516346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:49.523683 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:49.524134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:49.524866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:49.525333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:49.668168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:49.679328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:49.680852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:49.682117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-51da3b0a-67a5-4ab5-9a5c-af4db838ed50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.170828 May 25 16:20:49.682645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 248/486] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:50.700244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:50.700796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:50.707711 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:50.708154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:50.708898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:50.709329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:50.844458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:50.853489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:50.854993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:50.856376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-baa67296-5c7b-42b7-8d21-63de0923c156 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1809 microversion: 2.20 time: 0.160601 May 25 16:20:50.856999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 239/487] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:50 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1809 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:20:51.879235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:51.880396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:51.892728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:51.893477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:51.894643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:51.895092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:52.034883 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:52.043757 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:52.045253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:52.046405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-adf30735-c6f2-4ae4-995c-1a1d7f22cccf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1796 microversion: 2.20 time: 0.174167 May 25 16:20:52.046994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 249/488] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:20:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1796 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:20:54.189696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-17e7e58b-07c6-4609-8586-61023155f6ac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:54.190219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-17e7e58b-07c6-4609-8586-61023155f6ac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:54.192631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=22 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:54.198315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-17e7e58b-07c6-4609-8586-61023155f6ac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:54.198768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-17e7e58b-07c6-4609-8586-61023155f6ac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:54.199566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-17e7e58b-07c6-4609-8586-61023155f6ac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:54.199998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-17e7e58b-07c6-4609-8586-61023155f6ac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:54.335955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-17e7e58b-07c6-4609-8586-61023155f6ac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lazy-loading 'services' on Instance uuid 516581ca-a13c-4a28-92da-988be630b29c {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:20:54.466743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-17e7e58b-07c6-4609-8586-61023155f6ac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "DELETE /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c/os-volume_attachments/e5dd7a55-9b8e-4463-83a8-1b7cc4540179" status: 202 len: 0 microversion: 2.1 time: 0.280410 May 25 16:20:54.467589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 240/489] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:20:54 2022] DELETE /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c/os-volume_attachments/e5dd7a55-9b8e-4463-83a8-1b7cc4540179 => generated 0 bytes in 283 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:20:56.648630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9b709904-4176-4308-8a1a-55ff4ecdff2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:56.649151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9b709904-4176-4308-8a1a-55ff4ecdff2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:56.651660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:56.657003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b709904-4176-4308-8a1a-55ff4ecdff2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:56.657408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b709904-4176-4308-8a1a-55ff4ecdff2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:56.658183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b709904-4176-4308-8a1a-55ff4ecdff2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:56.658582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b709904-4176-4308-8a1a-55ff4ecdff2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:56.685447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9b709904-4176-4308-8a1a-55ff4ecdff2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:20:56.774362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9b709904-4176-4308-8a1a-55ff4ecdff2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "DELETE /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 204 len: 0 microversion: 2.1 time: 0.129247 May 25 16:20:56.774955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 250/490] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:20:56 2022] DELETE /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:20:56.786847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-65d46d33-0283-4470-a060-7a88cf2919f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:56.787572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-65d46d33-0283-4470-a060-7a88cf2919f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:56.794811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-65d46d33-0283-4470-a060-7a88cf2919f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:56.795290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-65d46d33-0283-4470-a060-7a88cf2919f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:56.796052 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-65d46d33-0283-4470-a060-7a88cf2919f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:56.796485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-65d46d33-0283-4470-a060-7a88cf2919f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:56.929025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-65d46d33-0283-4470-a060-7a88cf2919f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:56.936728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-65d46d33-0283-4470-a060-7a88cf2919f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1512 microversion: 2.1 time: 0.156566 May 25 16:20:56.937301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 241/491] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:20:56 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1512 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:57.958538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-371198de-85bc-444d-b6ef-aa62bdc30406 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:57.959440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-371198de-85bc-444d-b6ef-aa62bdc30406 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:57.966893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-371198de-85bc-444d-b6ef-aa62bdc30406 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:57.967477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-371198de-85bc-444d-b6ef-aa62bdc30406 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:57.968375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-371198de-85bc-444d-b6ef-aa62bdc30406 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:57.968926 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-371198de-85bc-444d-b6ef-aa62bdc30406 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:58.110567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-371198de-85bc-444d-b6ef-aa62bdc30406 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:58.118707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-371198de-85bc-444d-b6ef-aa62bdc30406 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1512 microversion: 2.1 time: 0.166884 May 25 16:20:58.119302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 251/492] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:20:57 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1512 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:59.136324 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-eda1c7a0-b15e-42a6-a62e-1eccab55a791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:20:59.136912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-eda1c7a0-b15e-42a6-a62e-1eccab55a791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:20:59.144049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eda1c7a0-b15e-42a6-a62e-1eccab55a791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:59.144515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eda1c7a0-b15e-42a6-a62e-1eccab55a791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:59.145264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eda1c7a0-b15e-42a6-a62e-1eccab55a791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:59.145726 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eda1c7a0-b15e-42a6-a62e-1eccab55a791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:59.353203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-eda1c7a0-b15e-42a6-a62e-1eccab55a791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:20:59.361159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-eda1c7a0-b15e-42a6-a62e-1eccab55a791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 200 len: 1512 microversion: 2.1 time: 0.228812 May 25 16:20:59.361953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 242/493] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:20:59 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 1512 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:20:59.521154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-0c86ab81-ac64-4472-8043-2858f82dc169 req-3f40a28f-944f-4c8b-ad0a-96ef16ff0ab9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "516581ca-a13c-4a28-92da-988be630b29c", "name": "network-vif-deleted", "tag": "798e662c-998a-4c12-afb9-3d2641d4ff07"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:20:59.530481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-0c86ab81-ac64-4472-8043-2858f82dc169 req-3f40a28f-944f-4c8b-ad0a-96ef16ff0ab9 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:20:59.530920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-0c86ab81-ac64-4472-8043-2858f82dc169 req-3f40a28f-944f-4c8b-ad0a-96ef16ff0ab9 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:20:59.542491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-0c86ab81-ac64-4472-8043-2858f82dc169 req-3f40a28f-944f-4c8b-ad0a-96ef16ff0ab9 service nova] Creating event network-vif-deleted:798e662c-998a-4c12-afb9-3d2641d4ff07 for instance 516581ca-a13c-4a28-92da-988be630b29c on hv-cinder-84327 May 25 16:20:59.547746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-0c86ab81-ac64-4472-8043-2858f82dc169 req-3f40a28f-944f-4c8b-ad0a-96ef16ff0ab9 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030349 May 25 16:20:59.548685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 252/494] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:20:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:00.379457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:00.380048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 516581ca-a13c-4a28-92da-988be630b29c] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:00.387044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:00.387532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:00.388322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:00.388774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:00.402304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] HTTP exception thrown: Instance 516581ca-a13c-4a28-92da-988be630b29c could not be found. May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Request method failure captured: May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c HTTP/1.1 May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlcTScjWaWhWZ_Y2deTS_iB2n7fhEPm9xpcRCdPE-r6ZMxY0ZuDpSToYfeI_E_5xfVHNIs52uYE6eqLoLyVrCNCdQhvtSm-zE7RKP38WrqQwdjdKcvzjavXk0OYwhbdCOQ2Ef36er8czPOW__xeJxnmKi5Ep7buFXy2Oju0Jbg9THQA May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:21:00.403999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_073368e447af465abb41aa556ee46dc9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/073368e447af465abb41aa556ee46dc9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: b53b45a485b14e0b9273c019dc9ec206 May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 516581ca-a13c-4a28-92da-988be630b29c could not be found. May 25 16:21:00.405444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '516581ca-a13c-4a28-92da-988be630b29c'} May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 516581ca-a13c-4a28-92da-988be630b29c could not be found. May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:21:00.406244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 516581ca-a13c-4a28-92da-988be630b29c could not be found. May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 516581ca-a13c-4a28-92da-988be630b29c could not be found. May 25 16:21:00.407463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:00.409017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Returning 404 to user: Instance 516581ca-a13c-4a28-92da-988be630b29c could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:21:00.409017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-aa24e99a-14b2-4bd2-a10d-94f03949a4de tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c" status: 404 len: 111 microversion: 2.1 time: 0.030432 May 25 16:21:00.409017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 243/495] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:00 2022] GET /compute/v2.1/servers/516581ca-a13c-4a28-92da-988be630b29c => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:21:03.615684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-83b2f3d1-e326-4912-8172-fda62c63928d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:03.616632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-83b2f3d1-e326-4912-8172-fda62c63928d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:03.623067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83b2f3d1-e326-4912-8172-fda62c63928d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:03.623540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83b2f3d1-e326-4912-8172-fda62c63928d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:03.624285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83b2f3d1-e326-4912-8172-fda62c63928d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:03.624706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83b2f3d1-e326-4912-8172-fda62c63928d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:03.750034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-83b2f3d1-e326-4912-8172-fda62c63928d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lazy-loading 'services' on Instance uuid f5ce685f-a7e1-41d6-96f4-e5cc19cad00a {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:21:03.867846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-83b2f3d1-e326-4912-8172-fda62c63928d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "DELETE /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/os-volume_attachments/b546e80a-6961-4be5-8ae2-198ede1fbbf9" status: 202 len: 0 microversion: 2.1 time: 0.260492 May 25 16:21:03.871189 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 253/496] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:21:03 2022] DELETE /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/os-volume_attachments/b546e80a-6961-4be5-8ae2-198ede1fbbf9 => generated 0 bytes in 265 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:21:04.992910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d5d5a1c9-cdb6-4066-99a9-6d5e16117e92 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:05.000052 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d5d5a1c9-cdb6-4066-99a9-6d5e16117e92 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:05.000502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d5d5a1c9-cdb6-4066-99a9-6d5e16117e92 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:05.063913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d5d5a1c9-cdb6-4066-99a9-6d5e16117e92 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Going to try to stop instance {{(pid=117795) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 25 16:21:05.112644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d5d5a1c9-cdb6-4066-99a9-6d5e16117e92 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/action" status: 202 len: 0 microversion: 2.1 time: 0.123564 May 25 16:21:05.113265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 244/497] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:21:04 2022] POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/action => generated 0 bytes in 125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:21:05.122823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e89f6094-f98a-4966-b672-e8fd22339047 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:05.123646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e89f6094-f98a-4966-b672-e8fd22339047 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:05.134408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e89f6094-f98a-4966-b672-e8fd22339047 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:05.134956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e89f6094-f98a-4966-b672-e8fd22339047 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:05.135839 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e89f6094-f98a-4966-b672-e8fd22339047 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:05.136278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e89f6094-f98a-4966-b672-e8fd22339047 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:05.271824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e89f6094-f98a-4966-b672-e8fd22339047 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:05.279608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e89f6094-f98a-4966-b672-e8fd22339047 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.161629 May 25 16:21:05.280148 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 254/498] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:05 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:06.303382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5607ee5d-56a3-44d5-aba5-8829e4cf0cfc tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:06.304560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5607ee5d-56a3-44d5-aba5-8829e4cf0cfc tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:06.315385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5607ee5d-56a3-44d5-aba5-8829e4cf0cfc tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:06.316118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5607ee5d-56a3-44d5-aba5-8829e4cf0cfc tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:06.317354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5607ee5d-56a3-44d5-aba5-8829e4cf0cfc tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:06.318072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5607ee5d-56a3-44d5-aba5-8829e4cf0cfc tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:06.453453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5607ee5d-56a3-44d5-aba5-8829e4cf0cfc tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:06.466067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5607ee5d-56a3-44d5-aba5-8829e4cf0cfc tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.170328 May 25 16:21:06.466970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 245/499] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:06 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:07.491026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e421151c-19ca-4a40-8d9f-844ffbd2d141 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:07.491805 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e421151c-19ca-4a40-8d9f-844ffbd2d141 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:07.498979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e421151c-19ca-4a40-8d9f-844ffbd2d141 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:07.499468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e421151c-19ca-4a40-8d9f-844ffbd2d141 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:07.500233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e421151c-19ca-4a40-8d9f-844ffbd2d141 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:07.500666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e421151c-19ca-4a40-8d9f-844ffbd2d141 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:07.642764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e421151c-19ca-4a40-8d9f-844ffbd2d141 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:07.651043 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e421151c-19ca-4a40-8d9f-844ffbd2d141 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.166487 May 25 16:21:07.651650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 255/500] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:07 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:08.677872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b0933f83-7aa8-4ef8-a192-6d977a14c212 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:08.678823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b0933f83-7aa8-4ef8-a192-6d977a14c212 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:08.689597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0933f83-7aa8-4ef8-a192-6d977a14c212 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:08.690338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0933f83-7aa8-4ef8-a192-6d977a14c212 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:08.691632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0933f83-7aa8-4ef8-a192-6d977a14c212 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:08.692346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0933f83-7aa8-4ef8-a192-6d977a14c212 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:08.833110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b0933f83-7aa8-4ef8-a192-6d977a14c212 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:08.841128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b0933f83-7aa8-4ef8-a192-6d977a14c212 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.170025 May 25 16:21:08.841697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 246/501] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:08 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:09.312414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-602933469", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "networks": [{"uuid": "76fa603d-f376-4c17-aeff-adef7293c5c4"}]}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:09.365429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.network.neutron [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] validate_networks() for [('76fa603d-f376-4c17-aeff-adef7293c5c4', None, None, None, None, None)] {{(pid=117794) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:21:09.859518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-98f03fd1-d7c5-45ba-9dc1-7bb5b218a9bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:09.860119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-98f03fd1-d7c5-45ba-9dc1-7bb5b218a9bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:09.866937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98f03fd1-d7c5-45ba-9dc1-7bb5b218a9bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:09.867397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98f03fd1-d7c5-45ba-9dc1-7bb5b218a9bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:09.868220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98f03fd1-d7c5-45ba-9dc1-7bb5b218a9bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:09.868685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98f03fd1-d7c5-45ba-9dc1-7bb5b218a9bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:09.927717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Flavor limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:21:09.928386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Image limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:21:09.928937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Flavor pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:21:09.929369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Image pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:21:09.930064 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:21:09.939561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:09.940189 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:09.941004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:09.941443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:09.959008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Getting quotas for project 073368e447af465abb41aa556ee46dc9. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:21:09.963468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Getting quotas for user b53b45a485b14e0b9273c019dc9ec206 and project 073368e447af465abb41aa556ee46dc9. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:21:09.975407 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Going to run 1 instances... {{(pid=117794) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:21:09.977740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117794) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:21:10.023582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fd860eee-6259-4efa-aaaa-af7a4f35ace6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.714905 May 25 16:21:10.024282 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 256/502] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:21:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 716 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:21:10.034206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e1a12415-735f-4a76-ac30-d61bbf55b0a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:10.034968 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e1a12415-735f-4a76-ac30-d61bbf55b0a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:10.093754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-98f03fd1-d7c5-45ba-9dc1-7bb5b218a9bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:10.101767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-98f03fd1-d7c5-45ba-9dc1-7bb5b218a9bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.245973 May 25 16:21:10.111394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 247/503] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:09 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:10.118563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e1a12415-735f-4a76-ac30-d61bbf55b0a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:10.125888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e1a12415-735f-4a76-ac30-d61bbf55b0a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1218 microversion: 2.1 time: 0.095529 May 25 16:21:10.126463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 257/504] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:10 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1218 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:11.119979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-207a7b1e-a021-4af8-bcc0-c9e1a99eb67e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:11.120625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-207a7b1e-a021-4af8-bcc0-c9e1a99eb67e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:11.128161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-207a7b1e-a021-4af8-bcc0-c9e1a99eb67e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:11.128610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-207a7b1e-a021-4af8-bcc0-c9e1a99eb67e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:11.129377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-207a7b1e-a021-4af8-bcc0-c9e1a99eb67e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:11.129941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-207a7b1e-a021-4af8-bcc0-c9e1a99eb67e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:11.142951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2d8c942a-e7d7-4f83-9f29-9280affcd776 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:11.143602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2d8c942a-e7d7-4f83-9f29-9280affcd776 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:11.150676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d8c942a-e7d7-4f83-9f29-9280affcd776 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:11.151192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d8c942a-e7d7-4f83-9f29-9280affcd776 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:11.152050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d8c942a-e7d7-4f83-9f29-9280affcd776 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:11.152545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d8c942a-e7d7-4f83-9f29-9280affcd776 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:11.227055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2d8c942a-e7d7-4f83-9f29-9280affcd776 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:11.235027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2d8c942a-e7d7-4f83-9f29-9280affcd776 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1270 microversion: 2.1 time: 0.095860 May 25 16:21:11.245933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 258/505] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:11 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1270 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:11.290040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-207a7b1e-a021-4af8-bcc0-c9e1a99eb67e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:11.298314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-207a7b1e-a021-4af8-bcc0-c9e1a99eb67e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.182880 May 25 16:21:11.298879 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 248/506] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:11 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:12.253639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-19cc9d52-34ba-4615-9b50-8fd1eefda323 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:12.254200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-19cc9d52-34ba-4615-9b50-8fd1eefda323 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:12.261147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19cc9d52-34ba-4615-9b50-8fd1eefda323 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:12.261609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19cc9d52-34ba-4615-9b50-8fd1eefda323 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:12.262362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19cc9d52-34ba-4615-9b50-8fd1eefda323 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:12.262775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19cc9d52-34ba-4615-9b50-8fd1eefda323 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:12.317893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ac5ccddd-a908-4b45-8b30-7004e0e7b40a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:12.318576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ac5ccddd-a908-4b45-8b30-7004e0e7b40a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:12.325711 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac5ccddd-a908-4b45-8b30-7004e0e7b40a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:12.326305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac5ccddd-a908-4b45-8b30-7004e0e7b40a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:12.327275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac5ccddd-a908-4b45-8b30-7004e0e7b40a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:12.327719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac5ccddd-a908-4b45-8b30-7004e0e7b40a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:12.815783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-19cc9d52-34ba-4615-9b50-8fd1eefda323 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:12.823757 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-19cc9d52-34ba-4615-9b50-8fd1eefda323 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1318 microversion: 2.1 time: 0.573976 May 25 16:21:12.824337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 259/507] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:12 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1318 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:12.845558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ac5ccddd-a908-4b45-8b30-7004e0e7b40a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:12.865227 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ac5ccddd-a908-4b45-8b30-7004e0e7b40a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.551415 May 25 16:21:12.866165 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 249/508] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:12 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:21:13.261510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-0525e165-3c48-41ac-a6a4-63b9e9eab5a7 req-b583d5d4-3cf4-4713-8bde-1372217c95d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3fc5dcaa-3275-47eb-88ec-62b23505c91b", "tag": "e6435373-0460-4e82-8377-2ccbc0dfa40a"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:13.271116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-0525e165-3c48-41ac-a6a4-63b9e9eab5a7 req-b583d5d4-3cf4-4713-8bde-1372217c95d2 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:13.271685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-0525e165-3c48-41ac-a6a4-63b9e9eab5a7 req-b583d5d4-3cf4-4713-8bde-1372217c95d2 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:13.293499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-0525e165-3c48-41ac-a6a4-63b9e9eab5a7 req-b583d5d4-3cf4-4713-8bde-1372217c95d2 service nova] Creating event network-changed:e6435373-0460-4e82-8377-2ccbc0dfa40a for instance 3fc5dcaa-3275-47eb-88ec-62b23505c91b on hv-cinder-84327 May 25 16:21:13.298430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-0525e165-3c48-41ac-a6a4-63b9e9eab5a7 req-b583d5d4-3cf4-4713-8bde-1372217c95d2 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040745 May 25 16:21:13.299186 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 260/509] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:21:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:13.845201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-628a8076-b26e-4d53-a20f-5d3c2f15ab0c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:13.846236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-628a8076-b26e-4d53-a20f-5d3c2f15ab0c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:13.855173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-628a8076-b26e-4d53-a20f-5d3c2f15ab0c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:13.855629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-628a8076-b26e-4d53-a20f-5d3c2f15ab0c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:13.856386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-628a8076-b26e-4d53-a20f-5d3c2f15ab0c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:13.856827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-628a8076-b26e-4d53-a20f-5d3c2f15ab0c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:13.884335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-726b42b8-16a4-46bf-8651-fd6d24343001 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:13.885211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-726b42b8-16a4-46bf-8651-fd6d24343001 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:13.892517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-726b42b8-16a4-46bf-8651-fd6d24343001 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:13.892967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-726b42b8-16a4-46bf-8651-fd6d24343001 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:13.893746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-726b42b8-16a4-46bf-8651-fd6d24343001 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:13.894182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-726b42b8-16a4-46bf-8651-fd6d24343001 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:14.053372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-628a8076-b26e-4d53-a20f-5d3c2f15ab0c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:14.062408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-628a8076-b26e-4d53-a20f-5d3c2f15ab0c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1318 microversion: 2.1 time: 0.223785 May 25 16:21:14.063824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 250/510] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:13 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1318 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:14.085068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-726b42b8-16a4-46bf-8651-fd6d24343001 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:14.093861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-726b42b8-16a4-46bf-8651-fd6d24343001 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.215225 May 25 16:21:14.094424 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 261/511] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:13 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:15.083912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7bcc1b98-1871-4c66-9794-74b7eb775e93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:15.084615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7bcc1b98-1871-4c66-9794-74b7eb775e93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:15.091654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7bcc1b98-1871-4c66-9794-74b7eb775e93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:15.092251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7bcc1b98-1871-4c66-9794-74b7eb775e93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:15.093127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7bcc1b98-1871-4c66-9794-74b7eb775e93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:15.093686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7bcc1b98-1871-4c66-9794-74b7eb775e93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:15.110772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c00ecbc3-f1a5-4b27-a7c7-cb9d3b33410c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:15.111371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c00ecbc3-f1a5-4b27-a7c7-cb9d3b33410c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:15.118994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c00ecbc3-f1a5-4b27-a7c7-cb9d3b33410c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:15.119571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c00ecbc3-f1a5-4b27-a7c7-cb9d3b33410c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:15.120361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c00ecbc3-f1a5-4b27-a7c7-cb9d3b33410c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:15.120787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c00ecbc3-f1a5-4b27-a7c7-cb9d3b33410c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:15.369374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7bcc1b98-1871-4c66-9794-74b7eb775e93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:15.377641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c00ecbc3-f1a5-4b27-a7c7-cb9d3b33410c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:15.379947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7bcc1b98-1871-4c66-9794-74b7eb775e93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1318 microversion: 2.1 time: 0.299748 May 25 16:21:15.380661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 251/512] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:15 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1318 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:15.387110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c00ecbc3-f1a5-4b27-a7c7-cb9d3b33410c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.280129 May 25 16:21:15.387695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 262/513] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:15 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:16.400456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3e931149-b6de-476d-ab0d-a21958cf706c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:16.401120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3e931149-b6de-476d-ab0d-a21958cf706c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:16.404373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-93befbfd-8084-4924-b798-04a378a10600 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:16.405004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-93befbfd-8084-4924-b798-04a378a10600 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:16.407970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e931149-b6de-476d-ab0d-a21958cf706c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:16.408416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e931149-b6de-476d-ab0d-a21958cf706c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:16.409179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e931149-b6de-476d-ab0d-a21958cf706c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:16.409610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e931149-b6de-476d-ab0d-a21958cf706c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:16.411451 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-93befbfd-8084-4924-b798-04a378a10600 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:16.411906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-93befbfd-8084-4924-b798-04a378a10600 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:16.412326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:16.412686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-93befbfd-8084-4924-b798-04a378a10600 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:16.413122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-93befbfd-8084-4924-b798-04a378a10600 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:16.616427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3e931149-b6de-476d-ab0d-a21958cf706c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:16.619692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-93befbfd-8084-4924-b798-04a378a10600 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:16.627620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-93befbfd-8084-4924-b798-04a378a10600 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.226954 May 25 16:21:16.628174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 263/514] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:16 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:16.628353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3e931149-b6de-476d-ab0d-a21958cf706c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1318 microversion: 2.1 time: 0.233141 May 25 16:21:16.628864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 252/515] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:16 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1318 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:17.649012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c7a253b5-f743-47b9-8c45-c1995ed0db1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:17.649890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0d5cac0a-528e-4e71-ae81-8a26fd0ef4c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:17.650510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c7a253b5-f743-47b9-8c45-c1995ed0db1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:17.650693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0d5cac0a-528e-4e71-ae81-8a26fd0ef4c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:17.660546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d5cac0a-528e-4e71-ae81-8a26fd0ef4c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:17.660823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7a253b5-f743-47b9-8c45-c1995ed0db1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:17.661299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d5cac0a-528e-4e71-ae81-8a26fd0ef4c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:17.661555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7a253b5-f743-47b9-8c45-c1995ed0db1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:17.662613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d5cac0a-528e-4e71-ae81-8a26fd0ef4c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:17.662859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7a253b5-f743-47b9-8c45-c1995ed0db1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:17.663397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d5cac0a-528e-4e71-ae81-8a26fd0ef4c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:17.663635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7a253b5-f743-47b9-8c45-c1995ed0db1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:17.666770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:17.860071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0d5cac0a-528e-4e71-ae81-8a26fd0ef4c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:17.860341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c7a253b5-f743-47b9-8c45-c1995ed0db1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:17.868457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c7a253b5-f743-47b9-8c45-c1995ed0db1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.225923 May 25 16:21:17.869355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 264/516] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:17 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:17.871199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0d5cac0a-528e-4e71-ae81-8a26fd0ef4c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1318 microversion: 2.1 time: 0.228568 May 25 16:21:17.871782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 253/517] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:17 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1318 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:18.292453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-02c78134-e46d-45d1-8194-a8a0c036f361 req-b84ce1e8-365d-4822-b0fa-cd7b9cfffbe5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3fc5dcaa-3275-47eb-88ec-62b23505c91b", "name": "network-vif-plugged", "status": "completed", "tag": "e6435373-0460-4e82-8377-2ccbc0dfa40a"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:18.300946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-02c78134-e46d-45d1-8194-a8a0c036f361 req-b84ce1e8-365d-4822-b0fa-cd7b9cfffbe5 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:18.301424 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-02c78134-e46d-45d1-8194-a8a0c036f361 req-b84ce1e8-365d-4822-b0fa-cd7b9cfffbe5 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:18.313740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-02c78134-e46d-45d1-8194-a8a0c036f361 req-b84ce1e8-365d-4822-b0fa-cd7b9cfffbe5 service nova] Creating event network-vif-plugged:e6435373-0460-4e82-8377-2ccbc0dfa40a for instance 3fc5dcaa-3275-47eb-88ec-62b23505c91b on hv-cinder-84327 May 25 16:21:18.318483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-02c78134-e46d-45d1-8194-a8a0c036f361 req-b84ce1e8-365d-4822-b0fa-cd7b9cfffbe5 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030323 May 25 16:21:18.319472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 265/518] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:21:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:18.890131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-31fbfbcf-70b4-425a-8fac-e041e297dc2f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:18.891093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-31fbfbcf-70b4-425a-8fac-e041e297dc2f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:18.894987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-73eea4a8-209c-4382-8a17-541d20c6f03e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:18.896187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-73eea4a8-209c-4382-8a17-541d20c6f03e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:18.901338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31fbfbcf-70b4-425a-8fac-e041e297dc2f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:18.901793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31fbfbcf-70b4-425a-8fac-e041e297dc2f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:18.902580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31fbfbcf-70b4-425a-8fac-e041e297dc2f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:18.903016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31fbfbcf-70b4-425a-8fac-e041e297dc2f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:18.903588 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-73eea4a8-209c-4382-8a17-541d20c6f03e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:18.904040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-73eea4a8-209c-4382-8a17-541d20c6f03e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:18.904798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-73eea4a8-209c-4382-8a17-541d20c6f03e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:18.905379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-73eea4a8-209c-4382-8a17-541d20c6f03e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:19.096181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-31fbfbcf-70b4-425a-8fac-e041e297dc2f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:19.099552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-73eea4a8-209c-4382-8a17-541d20c6f03e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:19.104028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-31fbfbcf-70b4-425a-8fac-e041e297dc2f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.220743 May 25 16:21:19.104601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 254/519] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:18 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:19.107467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-73eea4a8-209c-4382-8a17-541d20c6f03e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1318 microversion: 2.1 time: 0.221294 May 25 16:21:19.108029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 266/520] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:18 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1318 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:20.121569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8d4bd6c6-85de-44b8-88b1-e68d6d0a2ff8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:20.122188 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8d4bd6c6-85de-44b8-88b1-e68d6d0a2ff8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:20.125041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0a39b7d9-68b5-40d8-8ff2-232dd487ad83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:20.125651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0a39b7d9-68b5-40d8-8ff2-232dd487ad83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:20.128097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:20.128908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=8 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:20.129065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8d4bd6c6-85de-44b8-88b1-e68d6d0a2ff8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:20.129540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8d4bd6c6-85de-44b8-88b1-e68d6d0a2ff8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:20.130389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8d4bd6c6-85de-44b8-88b1-e68d6d0a2ff8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:20.130855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8d4bd6c6-85de-44b8-88b1-e68d6d0a2ff8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:20.134501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a39b7d9-68b5-40d8-8ff2-232dd487ad83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:20.134946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a39b7d9-68b5-40d8-8ff2-232dd487ad83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:20.135758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a39b7d9-68b5-40d8-8ff2-232dd487ad83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:20.136170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a39b7d9-68b5-40d8-8ff2-232dd487ad83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:20.324024 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8d4bd6c6-85de-44b8-88b1-e68d6d0a2ff8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:20.329073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0a39b7d9-68b5-40d8-8ff2-232dd487ad83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:20.335634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8d4bd6c6-85de-44b8-88b1-e68d6d0a2ff8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1678 microversion: 2.1 time: 0.217558 May 25 16:21:20.336703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 255/521] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:20 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1678 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:20.337321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0a39b7d9-68b5-40d8-8ff2-232dd487ad83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1506 microversion: 2.1 time: 0.216866 May 25 16:21:20.338083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 267/522] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:20 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1506 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:20.353962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a3e37374-5cbd-4152-aaba-bfeac68d54e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ca477fe6-273b-4b1f-a801-7dc44a982861", "device": "/dev/sdb"}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:20.355413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a3e37374-5cbd-4152-aaba-bfeac68d54e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:20.362128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a3e37374-5cbd-4152-aaba-bfeac68d54e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:20.362580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a3e37374-5cbd-4152-aaba-bfeac68d54e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:20.363368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a3e37374-5cbd-4152-aaba-bfeac68d54e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:20.363789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a3e37374-5cbd-4152-aaba-bfeac68d54e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:20.743117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a3e37374-5cbd-4152-aaba-bfeac68d54e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "POST /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.392840 May 25 16:21:20.744122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 256/523] 10.222.0.45 () {60 vars in 1322 bytes} [Wed May 25 16:21:20 2022] POST /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments => generated 194 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:21:21.172977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-91381e19-52ca-4d34-8df5-e5a44a44c0ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:21.173944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-91381e19-52ca-4d34-8df5-e5a44a44c0ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:21.180811 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-91381e19-52ca-4d34-8df5-e5a44a44c0ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:21.181269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-91381e19-52ca-4d34-8df5-e5a44a44c0ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:21.182028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-91381e19-52ca-4d34-8df5-e5a44a44c0ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:21.182461 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-91381e19-52ca-4d34-8df5-e5a44a44c0ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:21.259736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-91381e19-52ca-4d34-8df5-e5a44a44c0ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "POST /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/action" status: 202 len: 0 microversion: 2.1 time: 0.090751 May 25 16:21:21.260366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 268/524] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:21:21 2022] POST /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/action => generated 0 bytes in 92 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:21:21.269088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-50b8462c-991f-49ea-bb76-1d12d6f0809d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:21.269817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-50b8462c-991f-49ea-bb76-1d12d6f0809d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:21.276636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50b8462c-991f-49ea-bb76-1d12d6f0809d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:21.277083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50b8462c-991f-49ea-bb76-1d12d6f0809d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:21.277853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50b8462c-991f-49ea-bb76-1d12d6f0809d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:21.278283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50b8462c-991f-49ea-bb76-1d12d6f0809d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:21.355973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ca8cdfc1-4d5a-470f-b545-561e39ebb1fd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:21.356593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ca8cdfc1-4d5a-470f-b545-561e39ebb1fd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:21.363084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca8cdfc1-4d5a-470f-b545-561e39ebb1fd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:21.363592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca8cdfc1-4d5a-470f-b545-561e39ebb1fd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:21.364621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca8cdfc1-4d5a-470f-b545-561e39ebb1fd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:21.365327 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca8cdfc1-4d5a-470f-b545-561e39ebb1fd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:21.572013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-50b8462c-991f-49ea-bb76-1d12d6f0809d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:21.580575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-50b8462c-991f-49ea-bb76-1d12d6f0809d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1751 microversion: 2.1 time: 0.315748 May 25 16:21:21.581138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 257/525] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:21 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1751 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:21.648205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ca8cdfc1-4d5a-470f-b545-561e39ebb1fd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:21.656416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ca8cdfc1-4d5a-470f-b545-561e39ebb1fd tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1655 microversion: 2.1 time: 0.306265 May 25 16:21:21.657073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 269/526] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:21 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1655 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:21:21.673551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2576b30a-cf0d-4234-81b8-9f00c05d2c78 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:21.680852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2576b30a-cf0d-4234-81b8-9f00c05d2c78 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:21.681322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2576b30a-cf0d-4234-81b8-9f00c05d2c78 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:21.703780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2576b30a-cf0d-4234-81b8-9f00c05d2c78 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Going to try to start instance {{(pid=117795) start /opt/stack/nova/nova/compute/api.py:2730}} May 25 16:21:21.749918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2576b30a-cf0d-4234-81b8-9f00c05d2c78 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/action" status: 202 len: 0 microversion: 2.1 time: 0.080062 May 25 16:21:21.750555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 258/527] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:21:21 2022] POST /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/action => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:21:21.759333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-82f532f8-ed78-49a4-93f7-60748cf1d978 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:21.759969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-82f532f8-ed78-49a4-93f7-60748cf1d978 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:21.766337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82f532f8-ed78-49a4-93f7-60748cf1d978 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:21.767034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82f532f8-ed78-49a4-93f7-60748cf1d978 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:21.767852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82f532f8-ed78-49a4-93f7-60748cf1d978 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:21.768288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82f532f8-ed78-49a4-93f7-60748cf1d978 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.007866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-82f532f8-ed78-49a4-93f7-60748cf1d978 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:22.016086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-82f532f8-ed78-49a4-93f7-60748cf1d978 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1664 microversion: 2.1 time: 0.261011 May 25 16:21:22.016772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 270/528] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:21 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1664 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:22.168137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-10a51739-257b-48ca-99ce-22ea2d43c8e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:22.168774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-10a51739-257b-48ca-99ce-22ea2d43c8e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:22.175778 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-10a51739-257b-48ca-99ce-22ea2d43c8e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.176287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-10a51739-257b-48ca-99ce-22ea2d43c8e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.177051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-10a51739-257b-48ca-99ce-22ea2d43c8e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.177486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-10a51739-257b-48ca-99ce-22ea2d43c8e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.201967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-10a51739-257b-48ca-99ce-22ea2d43c8e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462/ips" status: 200 len: 150 microversion: 2.1 time: 0.039262 May 25 16:21:22.207722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 259/529] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:21:22 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462/ips => generated 150 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:22.214022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3946a5c7-9a7d-4de6-9801-eb9817e8dd65"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:22.215124 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:22.222484 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.223264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.224366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.224807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.307167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Invalid volume: volume 3946a5c7-9a7d-4de6-9801-eb9817e8dd65 is already attached to instances: 48341208-161c-454c-b88e-56a30f695cb7 May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: POST /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462/os-volume_attachments HTTP/1.1 May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 74 May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:21:22.308967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {"volumeAttachment": {"volumeId": "3946a5c7-9a7d-4de6-9801-eb9817e8dd65"}} May 25 16:21:22.310287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid volume: volume 3946a5c7-9a7d-4de6-9801-eb9817e8dd65 is already attached to instances: 48341208-161c-454c-b88e-56a30f695cb7 May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': '667322c4-d9ca-4320-903f-112d84e5f462', 'body': {'volumeAttachment': {'volumeId': '3946a5c7-9a7d-4de6-9801-eb9817e8dd65'}}} May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 3946a5c7-9a7d-4de6-9801-eb9817e8dd65 is already attached to instances: 48341208-161c-454c-b88e-56a30f695cb7 May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:21:22.311008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 3946a5c7-9a7d-4de6-9801-eb9817e8dd65 is already attached to instances: 48341208-161c-454c-b88e-56a30f695cb7 May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:21:22.312264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:21:22.313652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:21:22.313652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:21:22.313652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:21:22.313652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 3946a5c7-9a7d-4de6-9801-eb9817e8dd65 is already attached to instances: 48341208-161c-454c-b88e-56a30f695cb7 May 25 16:21:22.313652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:21:22.313652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 400 to user: Invalid volume: volume 3946a5c7-9a7d-4de6-9801-eb9817e8dd65 is already attached to instances: 48341208-161c-454c-b88e-56a30f695cb7 {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:21:22.313652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8aca51e2-99d6-4a3f-916f-fc2cb3c2e35b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.101254 May 25 16:21:22.313652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 271/530] 10.222.0.45 () {60 vars in 1322 bytes} [Wed May 25 16:21:22 2022] POST /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462/os-volume_attachments => generated 174 bytes in 102 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:21:22.319338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c5324b98-6ba5-47a7-866c-6fea087f3ab8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:22.319813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c5324b98-6ba5-47a7-866c-6fea087f3ab8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:22.325924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5324b98-6ba5-47a7-866c-6fea087f3ab8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.326374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5324b98-6ba5-47a7-866c-6fea087f3ab8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.327130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5324b98-6ba5-47a7-866c-6fea087f3ab8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.327591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5324b98-6ba5-47a7-866c-6fea087f3ab8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.352216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c5324b98-6ba5-47a7-866c-6fea087f3ab8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Going to try to terminate instance {{(pid=117795) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:21:22.432358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c5324b98-6ba5-47a7-866c-6fea087f3ab8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 204 len: 0 microversion: 2.1 time: 0.116648 May 25 16:21:22.436011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 260/531] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:21:22 2022] DELETE /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 25 16:21:22.441253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-66096dcc-11f9-4a63-8fb7-679e28fded01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:22.441899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-66096dcc-11f9-4a63-8fb7-679e28fded01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:22.448331 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66096dcc-11f9-4a63-8fb7-679e28fded01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.448784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66096dcc-11f9-4a63-8fb7-679e28fded01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.449576 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66096dcc-11f9-4a63-8fb7-679e28fded01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.450036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66096dcc-11f9-4a63-8fb7-679e28fded01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.586377 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-66096dcc-11f9-4a63-8fb7-679e28fded01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:22.594093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-66096dcc-11f9-4a63-8fb7-679e28fded01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1681 microversion: 2.1 time: 0.156739 May 25 16:21:22.594659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 272/532] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:22 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1681 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:21:22.599127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-20f57811-8b38-4dee-988d-b61c1c57ae7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:22.599889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-20f57811-8b38-4dee-988d-b61c1c57ae7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:22.606429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20f57811-8b38-4dee-988d-b61c1c57ae7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.606874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20f57811-8b38-4dee-988d-b61c1c57ae7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.607689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20f57811-8b38-4dee-988d-b61c1c57ae7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:22.608120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20f57811-8b38-4dee-988d-b61c1c57ae7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:22.755901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-20f57811-8b38-4dee-988d-b61c1c57ae7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:22.764269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-20f57811-8b38-4dee-988d-b61c1c57ae7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.169521 May 25 16:21:22.764834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 261/533] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:22 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:23.037982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b60ed815-7c94-4370-9159-247ff9382366 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:23.039129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b60ed815-7c94-4370-9159-247ff9382366 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:23.046878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b60ed815-7c94-4370-9159-247ff9382366 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:23.047425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b60ed815-7c94-4370-9159-247ff9382366 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:23.048268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b60ed815-7c94-4370-9159-247ff9382366 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:23.048747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b60ed815-7c94-4370-9159-247ff9382366 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:23.184408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b60ed815-7c94-4370-9159-247ff9382366 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:23.192719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b60ed815-7c94-4370-9159-247ff9382366 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1664 microversion: 2.1 time: 0.161452 May 25 16:21:23.193556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 273/534] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:23 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1664 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:23.616849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7be15692-3d51-4eb8-bead-0ea3df72a30e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:23.617866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7be15692-3d51-4eb8-bead-0ea3df72a30e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:23.628912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7be15692-3d51-4eb8-bead-0ea3df72a30e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:23.629668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7be15692-3d51-4eb8-bead-0ea3df72a30e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:23.630926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7be15692-3d51-4eb8-bead-0ea3df72a30e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:23.631702 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7be15692-3d51-4eb8-bead-0ea3df72a30e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:23.778487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7be15692-3d51-4eb8-bead-0ea3df72a30e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:23.781941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-26542514-a018-48d2-abb4-db410b66975e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:23.782519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-26542514-a018-48d2-abb4-db410b66975e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:23.786534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7be15692-3d51-4eb8-bead-0ea3df72a30e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1681 microversion: 2.1 time: 0.176442 May 25 16:21:23.787110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 262/535] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:23 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1681 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:23.789331 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-26542514-a018-48d2-abb4-db410b66975e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:23.789782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-26542514-a018-48d2-abb4-db410b66975e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:23.790881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-26542514-a018-48d2-abb4-db410b66975e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:23.791332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-26542514-a018-48d2-abb4-db410b66975e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:23.947804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-26542514-a018-48d2-abb4-db410b66975e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:23.955965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-26542514-a018-48d2-abb4-db410b66975e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.177830 May 25 16:21:23.956554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 274/536] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:23 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:21:24.211984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9aa5ae37-876f-4311-82fa-64acdd5bfb4b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:24.212691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9aa5ae37-876f-4311-82fa-64acdd5bfb4b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:24.214818 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:24.220326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9aa5ae37-876f-4311-82fa-64acdd5bfb4b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:24.220788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9aa5ae37-876f-4311-82fa-64acdd5bfb4b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:24.221526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9aa5ae37-876f-4311-82fa-64acdd5bfb4b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:24.221953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9aa5ae37-876f-4311-82fa-64acdd5bfb4b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:24.359513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9aa5ae37-876f-4311-82fa-64acdd5bfb4b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:24.371616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9aa5ae37-876f-4311-82fa-64acdd5bfb4b tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1664 microversion: 2.1 time: 0.165182 May 25 16:21:24.372385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 263/537] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:24 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1664 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:24.805060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f14bcf7f-3647-45d2-bbb0-0f8ec86a74c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:24.805941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f14bcf7f-3647-45d2-bbb0-0f8ec86a74c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:24.813180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f14bcf7f-3647-45d2-bbb0-0f8ec86a74c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:24.813627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f14bcf7f-3647-45d2-bbb0-0f8ec86a74c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:24.814419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f14bcf7f-3647-45d2-bbb0-0f8ec86a74c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:24.814842 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f14bcf7f-3647-45d2-bbb0-0f8ec86a74c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:24.950405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f14bcf7f-3647-45d2-bbb0-0f8ec86a74c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:24.958495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f14bcf7f-3647-45d2-bbb0-0f8ec86a74c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1681 microversion: 2.1 time: 0.157758 May 25 16:21:24.959123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 275/538] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:24 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1681 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:24.972793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9657afe6-c62d-4341-b2b7-23056c3da99e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:24.973462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9657afe6-c62d-4341-b2b7-23056c3da99e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:24.980937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9657afe6-c62d-4341-b2b7-23056c3da99e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:24.981396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9657afe6-c62d-4341-b2b7-23056c3da99e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:24.982134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9657afe6-c62d-4341-b2b7-23056c3da99e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:24.982909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9657afe6-c62d-4341-b2b7-23056c3da99e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:25.129657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9657afe6-c62d-4341-b2b7-23056c3da99e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:25.138125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9657afe6-c62d-4341-b2b7-23056c3da99e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.169492 May 25 16:21:25.138679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 264/539] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:24 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:25.214472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e3760bb4-cc45-4891-a04a-0835ee180ddd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:25.216167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e3760bb4-cc45-4891-a04a-0835ee180ddd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:25.225579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3760bb4-cc45-4891-a04a-0835ee180ddd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:25.226450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3760bb4-cc45-4891-a04a-0835ee180ddd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:25.227765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3760bb4-cc45-4891-a04a-0835ee180ddd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:25.228271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3760bb4-cc45-4891-a04a-0835ee180ddd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:25.392355 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a17e9804-6dd2-4b92-a75c-0e92143194ec tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:25.392966 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a17e9804-6dd2-4b92-a75c-0e92143194ec tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:25.400358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a17e9804-6dd2-4b92-a75c-0e92143194ec tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:25.403330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a17e9804-6dd2-4b92-a75c-0e92143194ec tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:25.403330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a17e9804-6dd2-4b92-a75c-0e92143194ec tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:25.403330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a17e9804-6dd2-4b92-a75c-0e92143194ec tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:25.405516 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:25.509402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e3760bb4-cc45-4891-a04a-0835ee180ddd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.301317 May 25 16:21:25.516402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 276/540] 10.222.0.45 () {60 vars in 1322 bytes} [Wed May 25 16:21:25 2022] POST /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 194 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:25.568535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a17e9804-6dd2-4b92-a75c-0e92143194ec tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:25.578815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a17e9804-6dd2-4b92-a75c-0e92143194ec tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1668 microversion: 2.1 time: 0.190062 May 25 16:21:25.579891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 265/541] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:25 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1668 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:25.819880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-bfa975d2-348e-468f-91ed-fc158db4c3c3 req-4a43f9ef-4fbe-4b25-a2d9-2ee46e0284cd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "667322c4-d9ca-4320-903f-112d84e5f462", "name": "network-vif-deleted", "tag": "0906f7f3-63c4-42d4-9006-1ed37e306e92"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:21:25.829641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-bfa975d2-348e-468f-91ed-fc158db4c3c3 req-4a43f9ef-4fbe-4b25-a2d9-2ee46e0284cd service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:25.830165 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-bfa975d2-348e-468f-91ed-fc158db4c3c3 req-4a43f9ef-4fbe-4b25-a2d9-2ee46e0284cd service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:25.842153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-bfa975d2-348e-468f-91ed-fc158db4c3c3 req-4a43f9ef-4fbe-4b25-a2d9-2ee46e0284cd service nova] Creating event network-vif-deleted:0906f7f3-63c4-42d4-9006-1ed37e306e92 for instance 667322c4-d9ca-4320-903f-112d84e5f462 on hv-cinder-84327 May 25 16:21:25.847409 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-bfa975d2-348e-468f-91ed-fc158db4c3c3 req-4a43f9ef-4fbe-4b25-a2d9-2ee46e0284cd service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032236 May 25 16:21:25.848320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 277/542] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:21:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:25.976004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:25.976938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:25.983530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:25.984007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:25.985327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:25.985759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:26.018079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:26.018645 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:26.019092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lazy-loading 'fault' on Instance uuid 667322c4-d9ca-4320-903f-112d84e5f462 {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:21:26.067733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:26.075880 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1d4928f6-cf26-4a02-be67-5641dd8904d4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 200 len: 1331 microversion: 2.1 time: 0.103717 May 25 16:21:26.077079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 266/543] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:25 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 1331 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:26.156669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e7659a36-6e94-4e77-b455-24de46162f13 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:26.157586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e7659a36-6e94-4e77-b455-24de46162f13 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:26.164595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7659a36-6e94-4e77-b455-24de46162f13 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:26.165252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7659a36-6e94-4e77-b455-24de46162f13 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:26.166391 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7659a36-6e94-4e77-b455-24de46162f13 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:26.166899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7659a36-6e94-4e77-b455-24de46162f13 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:26.312045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e7659a36-6e94-4e77-b455-24de46162f13 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:26.324256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e7659a36-6e94-4e77-b455-24de46162f13 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.172228 May 25 16:21:26.325258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 278/544] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:26 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:27.094002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:27.094634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:27.102574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:27.103020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:27.103843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:27.104276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:27.119096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 HTTP/1.1 May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:21:27.121032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:21:27.122431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '667322c4-d9ca-4320-903f-112d84e5f462'} May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:21:27.123133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:21:27.124290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:21:27.125340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:21:27.125340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c4c55d97-e635-4d98-9ab8-b533d42c0c0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 404 len: 111 microversion: 2.1 time: 0.032625 May 25 16:21:27.125340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 267/545] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:27 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:21:27.131698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5fca45f8-3bd2-46e6-af80-d5de1cf7ec76 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '.version_select at 0x7f4a3004f280>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:27.138338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=76,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:27.154574 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5fca45f8-3bd2-46e6-af80-d5de1cf7ec76 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1616215641" status: 202 len: 0 microversion: 2.1 time: 0.026539 May 25 16:21:27.155476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 279/546] 10.222.0.45 () {58 vars in 1220 bytes} [Wed May 25 16:21:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1616215641 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:21:27.343339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f45d71ff-f426-464b-a63a-df6a9732f59f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:27.343967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f45d71ff-f426-464b-a63a-df6a9732f59f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:27.357758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f45d71ff-f426-464b-a63a-df6a9732f59f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:27.358285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f45d71ff-f426-464b-a63a-df6a9732f59f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:27.359145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f45d71ff-f426-464b-a63a-df6a9732f59f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:27.359658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f45d71ff-f426-464b-a63a-df6a9732f59f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:27.565684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f45d71ff-f426-464b-a63a-df6a9732f59f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:27.575429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f45d71ff-f426-464b-a63a-df6a9732f59f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.237250 May 25 16:21:27.576080 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 268/547] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:27 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:28.145874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-741deb2b-5e61-4da7-aff5-58b220828a51 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:28.146366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-741deb2b-5e61-4da7-aff5-58b220828a51 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:28.154318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-741deb2b-5e61-4da7-aff5-58b220828a51 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:28.154824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-741deb2b-5e61-4da7-aff5-58b220828a51 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:28.155672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-741deb2b-5e61-4da7-aff5-58b220828a51 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:28.156122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-741deb2b-5e61-4da7-aff5-58b220828a51 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:28.237700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:28.244451 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-741deb2b-5e61-4da7-aff5-58b220828a51 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lazy-loading 'services' on Instance uuid 48341208-161c-454c-b88e-56a30f695cb7 {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:21:28.319961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-741deb2b-5e61-4da7-aff5-58b220828a51 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments/3946a5c7-9a7d-4de6-9801-eb9817e8dd65" status: 202 len: 0 microversion: 2.1 time: 0.177768 May 25 16:21:28.321147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 280/548] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:21:28 2022] DELETE /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments/3946a5c7-9a7d-4de6-9801-eb9817e8dd65 => generated 0 bytes in 180 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:21:28.329404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-37569dd0-0717-4e94-bef8-3a6f43f473f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:28.330420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-37569dd0-0717-4e94-bef8-3a6f43f473f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:28.339651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-37569dd0-0717-4e94-bef8-3a6f43f473f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:28.340478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-37569dd0-0717-4e94-bef8-3a6f43f473f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:28.341995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-37569dd0-0717-4e94-bef8-3a6f43f473f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:28.342951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-37569dd0-0717-4e94-bef8-3a6f43f473f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:28.381691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-37569dd0-0717-4e94-bef8-3a6f43f473f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056301 May 25 16:21:28.382955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 269/549] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:28.594047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b8a6c113-9b28-4677-af47-65ad2cf785a5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:28.594737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b8a6c113-9b28-4677-af47-65ad2cf785a5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:28.601939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b8a6c113-9b28-4677-af47-65ad2cf785a5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:28.602466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b8a6c113-9b28-4677-af47-65ad2cf785a5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:28.603352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b8a6c113-9b28-4677-af47-65ad2cf785a5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:28.603778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b8a6c113-9b28-4677-af47-65ad2cf785a5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:28.759429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b8a6c113-9b28-4677-af47-65ad2cf785a5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:28.768178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b8a6c113-9b28-4677-af47-65ad2cf785a5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.178352 May 25 16:21:28.768858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 281/550] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:28 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:29.398167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-58ba4a7c-c5b1-4415-85bf-55116cb2b782 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:29.399095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-58ba4a7c-c5b1-4415-85bf-55116cb2b782 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:29.406305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58ba4a7c-c5b1-4415-85bf-55116cb2b782 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:29.406776 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58ba4a7c-c5b1-4415-85bf-55116cb2b782 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:29.407603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58ba4a7c-c5b1-4415-85bf-55116cb2b782 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:29.408069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58ba4a7c-c5b1-4415-85bf-55116cb2b782 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:29.441086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-58ba4a7c-c5b1-4415-85bf-55116cb2b782 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049866 May 25 16:21:29.442013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 270/551] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:29 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:29.788878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2ab5e816-3b31-41ce-b5c9-cd2d51e5e01e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:29.789647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2ab5e816-3b31-41ce-b5c9-cd2d51e5e01e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:29.797153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ab5e816-3b31-41ce-b5c9-cd2d51e5e01e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:29.797622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ab5e816-3b31-41ce-b5c9-cd2d51e5e01e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:29.798376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ab5e816-3b31-41ce-b5c9-cd2d51e5e01e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:29.798798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ab5e816-3b31-41ce-b5c9-cd2d51e5e01e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:29.955000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2ab5e816-3b31-41ce-b5c9-cd2d51e5e01e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:29.963593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2ab5e816-3b31-41ce-b5c9-cd2d51e5e01e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.180939 May 25 16:21:29.964149 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 282/552] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:29 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:30.456959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cee93959-9e18-4f16-8ca9-2c1d47f0a8d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:30.458454 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cee93959-9e18-4f16-8ca9-2c1d47f0a8d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:30.469944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cee93959-9e18-4f16-8ca9-2c1d47f0a8d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:30.470680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cee93959-9e18-4f16-8ca9-2c1d47f0a8d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:30.471983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cee93959-9e18-4f16-8ca9-2c1d47f0a8d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:30.472689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cee93959-9e18-4f16-8ca9-2c1d47f0a8d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:30.504293 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cee93959-9e18-4f16-8ca9-2c1d47f0a8d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053814 May 25 16:21:30.504847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 271/553] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:30 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:30.981408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-549d093a-c144-447b-9040-bd955fa46513 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:30.981971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-549d093a-c144-447b-9040-bd955fa46513 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:30.989146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-549d093a-c144-447b-9040-bd955fa46513 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:30.989600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-549d093a-c144-447b-9040-bd955fa46513 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:30.990364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-549d093a-c144-447b-9040-bd955fa46513 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:30.990802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-549d093a-c144-447b-9040-bd955fa46513 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:31.138560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-549d093a-c144-447b-9040-bd955fa46513 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:31.147308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-549d093a-c144-447b-9040-bd955fa46513 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.169739 May 25 16:21:31.148195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 283/554] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:30 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:31.517544 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-66ea1cd3-644b-44e7-a206-df25bb6ff479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:31.518725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-66ea1cd3-644b-44e7-a206-df25bb6ff479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:31.527586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-66ea1cd3-644b-44e7-a206-df25bb6ff479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:31.528100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-66ea1cd3-644b-44e7-a206-df25bb6ff479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:31.529195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-66ea1cd3-644b-44e7-a206-df25bb6ff479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:31.529775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-66ea1cd3-644b-44e7-a206-df25bb6ff479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:31.567765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-66ea1cd3-644b-44e7-a206-df25bb6ff479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055060 May 25 16:21:31.568342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 272/555] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:31 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:32.169937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8ec16174-1ca7-44b4-b51e-f348a06dfbad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:32.170800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8ec16174-1ca7-44b4-b51e-f348a06dfbad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:32.181509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ec16174-1ca7-44b4-b51e-f348a06dfbad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:32.181949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ec16174-1ca7-44b4-b51e-f348a06dfbad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:32.182727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ec16174-1ca7-44b4-b51e-f348a06dfbad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:32.183485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ec16174-1ca7-44b4-b51e-f348a06dfbad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:32.340344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8ec16174-1ca7-44b4-b51e-f348a06dfbad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:32.349117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8ec16174-1ca7-44b4-b51e-f348a06dfbad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.186235 May 25 16:21:32.349684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 284/556] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:32 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:32.579545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9b4b9984-1487-4316-be8e-413d5eba54bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:32.580472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9b4b9984-1487-4316-be8e-413d5eba54bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:32.587457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b4b9984-1487-4316-be8e-413d5eba54bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:32.587930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b4b9984-1487-4316-be8e-413d5eba54bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:32.588679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b4b9984-1487-4316-be8e-413d5eba54bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:32.589112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b4b9984-1487-4316-be8e-413d5eba54bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:32.618503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9b4b9984-1487-4316-be8e-413d5eba54bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042868 May 25 16:21:32.619302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 273/557] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:32 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:33.370244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-030c87f0-b9b5-4e6f-b90e-be4c75a8fa1d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:33.371236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-030c87f0-b9b5-4e6f-b90e-be4c75a8fa1d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:33.382149 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-030c87f0-b9b5-4e6f-b90e-be4c75a8fa1d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:33.382863 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-030c87f0-b9b5-4e6f-b90e-be4c75a8fa1d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:33.383850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-030c87f0-b9b5-4e6f-b90e-be4c75a8fa1d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:33.384287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-030c87f0-b9b5-4e6f-b90e-be4c75a8fa1d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:33.532367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-030c87f0-b9b5-4e6f-b90e-be4c75a8fa1d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:33.541230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-030c87f0-b9b5-4e6f-b90e-be4c75a8fa1d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.177713 May 25 16:21:33.541803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 285/558] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:33 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:33.636434 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-35758923-78cd-44d3-97cb-cb846c652eea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:33.638919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-35758923-78cd-44d3-97cb-cb846c652eea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:33.650754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-35758923-78cd-44d3-97cb-cb846c652eea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:33.651556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-35758923-78cd-44d3-97cb-cb846c652eea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:33.652330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-35758923-78cd-44d3-97cb-cb846c652eea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:33.652770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-35758923-78cd-44d3-97cb-cb846c652eea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:33.691520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-35758923-78cd-44d3-97cb-cb846c652eea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063473 May 25 16:21:33.692082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 274/559] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:33 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:34.553093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7d140698-fb8d-46d8-96d6-e1be6d6c20ae tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:34.553984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7d140698-fb8d-46d8-96d6-e1be6d6c20ae tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:34.560013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6d116bc1-f355-42af-9259-b8cd3e08a59b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:34.560640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6d116bc1-f355-42af-9259-b8cd3e08a59b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:34.560984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d140698-fb8d-46d8-96d6-e1be6d6c20ae tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.561411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d140698-fb8d-46d8-96d6-e1be6d6c20ae tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.562206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d140698-fb8d-46d8-96d6-e1be6d6c20ae tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.562646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d140698-fb8d-46d8-96d6-e1be6d6c20ae tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.567019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d116bc1-f355-42af-9259-b8cd3e08a59b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.567498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d116bc1-f355-42af-9259-b8cd3e08a59b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.568270 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d116bc1-f355-42af-9259-b8cd3e08a59b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.568675 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d116bc1-f355-42af-9259-b8cd3e08a59b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.593466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7d140698-fb8d-46d8-96d6-e1be6d6c20ae tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments/129ff07c-b9d7-4f99-9ec9-48c95b3d64ca" status: 200 len: 194 microversion: 2.20 time: 0.047248 May 25 16:21:34.594073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 286/560] 10.222.0.45 () {60 vars in 1453 bytes} [Wed May 25 16:21:34 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments/129ff07c-b9d7-4f99-9ec9-48c95b3d64ca => generated 194 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:34.658496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5f18055f-793d-47f7-bf9e-c7b63137cce8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:34.658991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5f18055f-793d-47f7-bf9e-c7b63137cce8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:34.665979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f18055f-793d-47f7-bf9e-c7b63137cce8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.666414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f18055f-793d-47f7-bf9e-c7b63137cce8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.667534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f18055f-793d-47f7-bf9e-c7b63137cce8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.667674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f18055f-793d-47f7-bf9e-c7b63137cce8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.709999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6d116bc1-f355-42af-9259-b8cd3e08a59b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:34.718834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6d116bc1-f355-42af-9259-b8cd3e08a59b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.162718 May 25 16:21:34.719404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 275/561] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:34 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:34.724006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-63212f2b-a9a8-44b3-ac96-c1a22239ae1c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:34.724924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-63212f2b-a9a8-44b3-ac96-c1a22239ae1c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:34.730831 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63212f2b-a9a8-44b3-ac96-c1a22239ae1c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.731321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63212f2b-a9a8-44b3-ac96-c1a22239ae1c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.732083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63212f2b-a9a8-44b3-ac96-c1a22239ae1c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.732518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63212f2b-a9a8-44b3-ac96-c1a22239ae1c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.752196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-5f18055f-793d-47f7-bf9e-c7b63137cce8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lazy-loading 'services' on Instance uuid 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:21:34.762304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-63212f2b-a9a8-44b3-ac96-c1a22239ae1c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041867 May 25 16:21:34.762873 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 276/562] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:34 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:34.825549 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5f18055f-793d-47f7-bf9e-c7b63137cce8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "DELETE /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments/129ff07c-b9d7-4f99-9ec9-48c95b3d64ca" status: 202 len: 0 microversion: 2.20 time: 0.170834 May 25 16:21:34.826129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 287/563] 10.222.0.45 () {60 vars in 1456 bytes} [Wed May 25 16:21:34 2022] DELETE /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments/129ff07c-b9d7-4f99-9ec9-48c95b3d64ca => generated 0 bytes in 172 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 25 16:21:34.834482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3cbf660d-84af-4b87-85cf-ca2a3bcac541 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:34.835472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3cbf660d-84af-4b87-85cf-ca2a3bcac541 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:34.841827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cbf660d-84af-4b87-85cf-ca2a3bcac541 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.842273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cbf660d-84af-4b87-85cf-ca2a3bcac541 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.843391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cbf660d-84af-4b87-85cf-ca2a3bcac541 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:34.844233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cbf660d-84af-4b87-85cf-ca2a3bcac541 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:34.873913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3cbf660d-84af-4b87-85cf-ca2a3bcac541 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043378 May 25 16:21:34.874483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 277/564] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:34 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:35.741542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0d4620c2-5113-460c-b76a-da925aa1c4db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:35.742740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0d4620c2-5113-460c-b76a-da925aa1c4db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:35.753411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d4620c2-5113-460c-b76a-da925aa1c4db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:35.754054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d4620c2-5113-460c-b76a-da925aa1c4db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:35.755333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d4620c2-5113-460c-b76a-da925aa1c4db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:35.756090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d4620c2-5113-460c-b76a-da925aa1c4db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:35.774116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c4912795-6289-430a-9bcc-abbab1a9e025 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:35.775076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c4912795-6289-430a-9bcc-abbab1a9e025 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:35.781979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4912795-6289-430a-9bcc-abbab1a9e025 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:35.782429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4912795-6289-430a-9bcc-abbab1a9e025 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:35.783196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4912795-6289-430a-9bcc-abbab1a9e025 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:35.783662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4912795-6289-430a-9bcc-abbab1a9e025 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:35.814855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c4912795-6289-430a-9bcc-abbab1a9e025 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044513 May 25 16:21:35.815572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 278/565] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:35 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:35.886879 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-820fe38b-f391-4636-b230-42a446580ca5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:35.887935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-820fe38b-f391-4636-b230-42a446580ca5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:35.894485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-820fe38b-f391-4636-b230-42a446580ca5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:35.894944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-820fe38b-f391-4636-b230-42a446580ca5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:35.895759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-820fe38b-f391-4636-b230-42a446580ca5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:35.896211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-820fe38b-f391-4636-b230-42a446580ca5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:35.902806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0d4620c2-5113-460c-b76a-da925aa1c4db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:35.911185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0d4620c2-5113-460c-b76a-da925aa1c4db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.177296 May 25 16:21:35.911742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 288/566] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:35 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:35.927173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-820fe38b-f391-4636-b230-42a446580ca5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045172 May 25 16:21:35.927733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 279/567] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:35 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:36.830340 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1a6547ee-b46f-40bf-b7b8-eed6a3f49741 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:36.832032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1a6547ee-b46f-40bf-b7b8-eed6a3f49741 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:36.840300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a6547ee-b46f-40bf-b7b8-eed6a3f49741 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:36.840748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a6547ee-b46f-40bf-b7b8-eed6a3f49741 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:36.841495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a6547ee-b46f-40bf-b7b8-eed6a3f49741 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:36.841896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a6547ee-b46f-40bf-b7b8-eed6a3f49741 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:36.875349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1a6547ee-b46f-40bf-b7b8-eed6a3f49741 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051636 May 25 16:21:36.875898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 289/568] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:36 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:36.929523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a7278635-8b17-4c74-b708-86f967e8a2fb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:36.930122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a7278635-8b17-4c74-b708-86f967e8a2fb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:36.936906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a7278635-8b17-4c74-b708-86f967e8a2fb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:36.937358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a7278635-8b17-4c74-b708-86f967e8a2fb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:36.938126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a7278635-8b17-4c74-b708-86f967e8a2fb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:36.938558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a7278635-8b17-4c74-b708-86f967e8a2fb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:36.938942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6a7999bc-5d71-4a9f-b354-3b0671d28bcb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:36.939915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6a7999bc-5d71-4a9f-b354-3b0671d28bcb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:36.946290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a7999bc-5d71-4a9f-b354-3b0671d28bcb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:36.946723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a7999bc-5d71-4a9f-b354-3b0671d28bcb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:36.947526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a7999bc-5d71-4a9f-b354-3b0671d28bcb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:36.948007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a7999bc-5d71-4a9f-b354-3b0671d28bcb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:36.977410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6a7999bc-5d71-4a9f-b354-3b0671d28bcb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042600 May 25 16:21:36.977989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 290/569] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:36 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:21:37.090345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a7278635-8b17-4c74-b708-86f967e8a2fb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:37.099704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a7278635-8b17-4c74-b708-86f967e8a2fb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.173967 May 25 16:21:37.100623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 280/570] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:36 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:37.887205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b2de6885-bfc2-4f3a-8100-f96c583cb9ef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:37.889276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b2de6885-bfc2-4f3a-8100-f96c583cb9ef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:37.896375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2de6885-bfc2-4f3a-8100-f96c583cb9ef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:37.896822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2de6885-bfc2-4f3a-8100-f96c583cb9ef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:37.897595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2de6885-bfc2-4f3a-8100-f96c583cb9ef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:37.898017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2de6885-bfc2-4f3a-8100-f96c583cb9ef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:37.929227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b2de6885-bfc2-4f3a-8100-f96c583cb9ef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045999 May 25 16:21:37.929761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 291/571] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:37 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:37.994249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5f54d212-3ec0-46ab-865f-26ce14d0e8b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:37.995849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5f54d212-3ec0-46ab-865f-26ce14d0e8b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:38.006785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5f54d212-3ec0-46ab-865f-26ce14d0e8b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:38.007616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5f54d212-3ec0-46ab-865f-26ce14d0e8b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:38.008582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5f54d212-3ec0-46ab-865f-26ce14d0e8b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:38.009011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5f54d212-3ec0-46ab-865f-26ce14d0e8b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:38.048411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5f54d212-3ec0-46ab-865f-26ce14d0e8b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060810 May 25 16:21:38.048972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 281/572] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:37 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:38.119644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b5e96960-300c-48ee-b6d9-e0ef18218b15 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:38.120531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b5e96960-300c-48ee-b6d9-e0ef18218b15 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:38.129423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5e96960-300c-48ee-b6d9-e0ef18218b15 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:38.129937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5e96960-300c-48ee-b6d9-e0ef18218b15 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:38.130969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5e96960-300c-48ee-b6d9-e0ef18218b15 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:38.131589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5e96960-300c-48ee-b6d9-e0ef18218b15 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:38.282542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b5e96960-300c-48ee-b6d9-e0ef18218b15 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:38.293228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b5e96960-300c-48ee-b6d9-e0ef18218b15 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.178456 May 25 16:21:38.293790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 292/573] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:38 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:38.941690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6c5751af-9c5b-41f7-bcec-ffd51b042ba3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:38.942616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6c5751af-9c5b-41f7-bcec-ffd51b042ba3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:38.949370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c5751af-9c5b-41f7-bcec-ffd51b042ba3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:38.949813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c5751af-9c5b-41f7-bcec-ffd51b042ba3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:38.950563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c5751af-9c5b-41f7-bcec-ffd51b042ba3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:38.950982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c5751af-9c5b-41f7-bcec-ffd51b042ba3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:38.953337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:38.982232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6c5751af-9c5b-41f7-bcec-ffd51b042ba3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044568 May 25 16:21:38.982786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 282/574] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:38 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:39.063562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e654facb-8c2c-462f-b340-87c7dffe1212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:39.064775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e654facb-8c2c-462f-b340-87c7dffe1212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:39.073986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e654facb-8c2c-462f-b340-87c7dffe1212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:39.074585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e654facb-8c2c-462f-b340-87c7dffe1212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:39.075640 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e654facb-8c2c-462f-b340-87c7dffe1212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:39.076218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e654facb-8c2c-462f-b340-87c7dffe1212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:39.108715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e654facb-8c2c-462f-b340-87c7dffe1212 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051114 May 25 16:21:39.109284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 293/575] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:39 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:39.314413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e0c36280-5c77-49c4-9e53-a47e3561a1e0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:39.315567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e0c36280-5c77-49c4-9e53-a47e3561a1e0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:39.324050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0c36280-5c77-49c4-9e53-a47e3561a1e0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:39.324498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0c36280-5c77-49c4-9e53-a47e3561a1e0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:39.325261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0c36280-5c77-49c4-9e53-a47e3561a1e0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:39.325695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0c36280-5c77-49c4-9e53-a47e3561a1e0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:39.478477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e0c36280-5c77-49c4-9e53-a47e3561a1e0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:39.491738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e0c36280-5c77-49c4-9e53-a47e3561a1e0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.183299 May 25 16:21:39.492542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 283/576] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:39 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:39.994131 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-67b5bf20-6286-4ca4-9835-cfbdd1968776 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:39.995059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-67b5bf20-6286-4ca4-9835-cfbdd1968776 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:40.001477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67b5bf20-6286-4ca4-9835-cfbdd1968776 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:40.001927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67b5bf20-6286-4ca4-9835-cfbdd1968776 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:40.002676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67b5bf20-6286-4ca4-9835-cfbdd1968776 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:40.003104 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67b5bf20-6286-4ca4-9835-cfbdd1968776 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:40.032529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-67b5bf20-6286-4ca4-9835-cfbdd1968776 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042339 May 25 16:21:40.033081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 294/577] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:39 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:40.121954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7f885a59-9116-4ffb-8792-8fea40472ef0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:40.123019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7f885a59-9116-4ffb-8792-8fea40472ef0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:40.130970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f885a59-9116-4ffb-8792-8fea40472ef0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:40.131877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f885a59-9116-4ffb-8792-8fea40472ef0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:40.133398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f885a59-9116-4ffb-8792-8fea40472ef0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:40.134144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f885a59-9116-4ffb-8792-8fea40472ef0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:40.168086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7f885a59-9116-4ffb-8792-8fea40472ef0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050695 May 25 16:21:40.168661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 284/578] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:40 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:40.514384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2bb4009f-e70a-4e14-9634-9d849e3ad1cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:40.514993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2bb4009f-e70a-4e14-9634-9d849e3ad1cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:40.521869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2bb4009f-e70a-4e14-9634-9d849e3ad1cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:40.522324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2bb4009f-e70a-4e14-9634-9d849e3ad1cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:40.523132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2bb4009f-e70a-4e14-9634-9d849e3ad1cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:40.524231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2bb4009f-e70a-4e14-9634-9d849e3ad1cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:40.670664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2bb4009f-e70a-4e14-9634-9d849e3ad1cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:40.679290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2bb4009f-e70a-4e14-9634-9d849e3ad1cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.168690 May 25 16:21:40.679867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 295/579] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:40 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:41.048341 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cb8d69c7-8475-46b2-9768-8ccab355325c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:41.049830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cb8d69c7-8475-46b2-9768-8ccab355325c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:41.058418 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb8d69c7-8475-46b2-9768-8ccab355325c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:41.058869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb8d69c7-8475-46b2-9768-8ccab355325c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:41.059660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb8d69c7-8475-46b2-9768-8ccab355325c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:41.060091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb8d69c7-8475-46b2-9768-8ccab355325c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:41.090417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cb8d69c7-8475-46b2-9768-8ccab355325c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048836 May 25 16:21:41.090975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 285/580] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:41 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:41.184935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-71c2f460-bfda-409a-bd7a-b8c54cb145e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:41.186230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-71c2f460-bfda-409a-bd7a-b8c54cb145e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:41.194253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-71c2f460-bfda-409a-bd7a-b8c54cb145e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:41.194693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-71c2f460-bfda-409a-bd7a-b8c54cb145e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:41.195476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-71c2f460-bfda-409a-bd7a-b8c54cb145e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:41.196061 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-71c2f460-bfda-409a-bd7a-b8c54cb145e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:41.198781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=98,nova_cell1:INSERT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:41.232716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-71c2f460-bfda-409a-bd7a-b8c54cb145e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054652 May 25 16:21:41.233194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 296/581] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:41.700812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-259676e5-0445-45b7-819d-b9dd4b07d8cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:41.701815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-259676e5-0445-45b7-819d-b9dd4b07d8cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:41.712061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-259676e5-0445-45b7-819d-b9dd4b07d8cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:41.712811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-259676e5-0445-45b7-819d-b9dd4b07d8cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:41.714059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-259676e5-0445-45b7-819d-b9dd4b07d8cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:41.714785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-259676e5-0445-45b7-819d-b9dd4b07d8cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:41.878210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-259676e5-0445-45b7-819d-b9dd4b07d8cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:41.888440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-259676e5-0445-45b7-819d-b9dd4b07d8cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.194311 May 25 16:21:41.889014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 286/582] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:41 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:42.102467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e520229e-0a16-4926-9b1f-1b559b6abd94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:42.103425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e520229e-0a16-4926-9b1f-1b559b6abd94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:42.110309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e520229e-0a16-4926-9b1f-1b559b6abd94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:42.110750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e520229e-0a16-4926-9b1f-1b559b6abd94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:42.111541 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e520229e-0a16-4926-9b1f-1b559b6abd94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:42.111961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e520229e-0a16-4926-9b1f-1b559b6abd94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:42.141758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e520229e-0a16-4926-9b1f-1b559b6abd94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043433 May 25 16:21:42.142322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 297/583] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:42 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:42.248811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-16075938-c93a-414e-a050-0baa217e6dfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:42.250306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-16075938-c93a-414e-a050-0baa217e6dfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:42.259854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-16075938-c93a-414e-a050-0baa217e6dfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:42.260304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-16075938-c93a-414e-a050-0baa217e6dfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:42.261069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-16075938-c93a-414e-a050-0baa217e6dfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:42.261499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-16075938-c93a-414e-a050-0baa217e6dfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:42.291644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-16075938-c93a-414e-a050-0baa217e6dfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049581 May 25 16:21:42.292203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 287/584] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:42.906332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6e9fee85-12e9-4628-a2d1-0621fd4bb3fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:42.906857 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6e9fee85-12e9-4628-a2d1-0621fd4bb3fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:42.913718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e9fee85-12e9-4628-a2d1-0621fd4bb3fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:42.914170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e9fee85-12e9-4628-a2d1-0621fd4bb3fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:42.915004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e9fee85-12e9-4628-a2d1-0621fd4bb3fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:42.915426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e9fee85-12e9-4628-a2d1-0621fd4bb3fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:43.062920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6e9fee85-12e9-4628-a2d1-0621fd4bb3fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:43.071533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6e9fee85-12e9-4628-a2d1-0621fd4bb3fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.169498 May 25 16:21:43.072138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 298/585] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:42 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:43.158292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c6ea3103-81a3-403d-befb-7a19b92186b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:43.159707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c6ea3103-81a3-403d-befb-7a19b92186b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:43.168119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6ea3103-81a3-403d-befb-7a19b92186b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:43.168571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6ea3103-81a3-403d-befb-7a19b92186b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:43.169334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6ea3103-81a3-403d-befb-7a19b92186b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:43.169773 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6ea3103-81a3-403d-befb-7a19b92186b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:43.199862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c6ea3103-81a3-403d-befb-7a19b92186b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048313 May 25 16:21:43.200422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 288/586] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:43 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:43.306512 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e19f3d76-9e61-4ab5-9582-9bc0ce077fd2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:43.307520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e19f3d76-9e61-4ab5-9582-9bc0ce077fd2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:43.314699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e19f3d76-9e61-4ab5-9582-9bc0ce077fd2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:43.315183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e19f3d76-9e61-4ab5-9582-9bc0ce077fd2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:43.316008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e19f3d76-9e61-4ab5-9582-9bc0ce077fd2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:43.316466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e19f3d76-9e61-4ab5-9582-9bc0ce077fd2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:43.351978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e19f3d76-9e61-4ab5-9582-9bc0ce077fd2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051025 May 25 16:21:43.352536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 299/587] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:44.090282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d6955f40-7b57-461e-9271-dc3279e657db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:44.090956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d6955f40-7b57-461e-9271-dc3279e657db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:44.097781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6955f40-7b57-461e-9271-dc3279e657db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:44.098236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6955f40-7b57-461e-9271-dc3279e657db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:44.099010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6955f40-7b57-461e-9271-dc3279e657db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:44.099480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6955f40-7b57-461e-9271-dc3279e657db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:44.212014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-626084ff-825f-4921-990b-e3b812b4309b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:44.212972 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-626084ff-825f-4921-990b-e3b812b4309b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:44.219282 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-626084ff-825f-4921-990b-e3b812b4309b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:44.219731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-626084ff-825f-4921-990b-e3b812b4309b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:44.220481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-626084ff-825f-4921-990b-e3b812b4309b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:44.220913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-626084ff-825f-4921-990b-e3b812b4309b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:44.251760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-626084ff-825f-4921-990b-e3b812b4309b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043490 May 25 16:21:44.251916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d6955f40-7b57-461e-9271-dc3279e657db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:44.255826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 300/588] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:44 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:44.260921 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d6955f40-7b57-461e-9271-dc3279e657db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.174458 May 25 16:21:44.261496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 289/589] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:44 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:21:44.367231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e66200a1-cdbb-4b39-ad45-56d4c54e4f7f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:44.368147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e66200a1-cdbb-4b39-ad45-56d4c54e4f7f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:44.375713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e66200a1-cdbb-4b39-ad45-56d4c54e4f7f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:44.376208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e66200a1-cdbb-4b39-ad45-56d4c54e4f7f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:44.377049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e66200a1-cdbb-4b39-ad45-56d4c54e4f7f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:44.377527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e66200a1-cdbb-4b39-ad45-56d4c54e4f7f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:44.412181 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e66200a1-cdbb-4b39-ad45-56d4c54e4f7f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051745 May 25 16:21:44.412724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 301/590] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:44 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:45.267899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c9fd3480-c969-4751-bcc4-e5e1b9c604f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:45.269450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c9fd3480-c969-4751-bcc4-e5e1b9c604f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:45.280891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c9fd3480-c969-4751-bcc4-e5e1b9c604f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:45.281552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d429c89b-232e-49ba-a163-d67020f0c7bc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:45.281811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c9fd3480-c969-4751-bcc4-e5e1b9c604f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:45.282558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d429c89b-232e-49ba-a163-d67020f0c7bc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:45.282918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c9fd3480-c969-4751-bcc4-e5e1b9c604f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:45.283684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c9fd3480-c969-4751-bcc4-e5e1b9c604f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:45.291525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d429c89b-232e-49ba-a163-d67020f0c7bc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:45.291973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d429c89b-232e-49ba-a163-d67020f0c7bc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:45.292733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d429c89b-232e-49ba-a163-d67020f0c7bc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:45.293170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d429c89b-232e-49ba-a163-d67020f0c7bc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:45.315507 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c9fd3480-c969-4751-bcc4-e5e1b9c604f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054271 May 25 16:21:45.316092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 290/591] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:45 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:45.424441 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-50fe934c-2df7-4d15-b5dd-97c68d01954e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:45.425392 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-50fe934c-2df7-4d15-b5dd-97c68d01954e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:45.431947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50fe934c-2df7-4d15-b5dd-97c68d01954e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:45.432428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50fe934c-2df7-4d15-b5dd-97c68d01954e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:45.433160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50fe934c-2df7-4d15-b5dd-97c68d01954e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:45.433630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50fe934c-2df7-4d15-b5dd-97c68d01954e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:45.439748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d429c89b-232e-49ba-a163-d67020f0c7bc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:45.448091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d429c89b-232e-49ba-a163-d67020f0c7bc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.172930 May 25 16:21:45.448674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 302/592] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:45 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:45.463902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-50fe934c-2df7-4d15-b5dd-97c68d01954e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043460 May 25 16:21:45.464463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 291/593] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:45 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:21:46.331629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-00d58ed8-0576-46ed-8a84-379b1a58e8c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:46.333243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-00d58ed8-0576-46ed-8a84-379b1a58e8c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:46.342004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00d58ed8-0576-46ed-8a84-379b1a58e8c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:46.342454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00d58ed8-0576-46ed-8a84-379b1a58e8c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:46.343217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00d58ed8-0576-46ed-8a84-379b1a58e8c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:46.343666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00d58ed8-0576-46ed-8a84-379b1a58e8c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:46.373164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-00d58ed8-0576-46ed-8a84-379b1a58e8c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048337 May 25 16:21:46.373750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 303/594] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:46 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:46.467276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-170e81a1-e3a4-4a46-a917-e94c2dfd8d67 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:46.468090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-170e81a1-e3a4-4a46-a917-e94c2dfd8d67 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:46.475695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-170e81a1-e3a4-4a46-a917-e94c2dfd8d67 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:46.476206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c6da3670-6036-4bb1-9ec8-edbde2d7f7c1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:46.476686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-170e81a1-e3a4-4a46-a917-e94c2dfd8d67 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:46.477044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-170e81a1-e3a4-4a46-a917-e94c2dfd8d67 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:46.477439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c6da3670-6036-4bb1-9ec8-edbde2d7f7c1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:46.477594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-170e81a1-e3a4-4a46-a917-e94c2dfd8d67 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:46.484941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6da3670-6036-4bb1-9ec8-edbde2d7f7c1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:46.485434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6da3670-6036-4bb1-9ec8-edbde2d7f7c1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:46.486268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6da3670-6036-4bb1-9ec8-edbde2d7f7c1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:46.486738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6da3670-6036-4bb1-9ec8-edbde2d7f7c1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:46.517054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c6da3670-6036-4bb1-9ec8-edbde2d7f7c1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045161 May 25 16:21:46.517597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 304/595] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:46 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:46.630326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-170e81a1-e3a4-4a46-a917-e94c2dfd8d67 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:46.638547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-170e81a1-e3a4-4a46-a917-e94c2dfd8d67 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.175643 May 25 16:21:46.639121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 292/596] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:46 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:47.389194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-86280737-ba00-45bb-a8f3-04da514474ac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:47.390685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-86280737-ba00-45bb-a8f3-04da514474ac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:47.400325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86280737-ba00-45bb-a8f3-04da514474ac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:47.400781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86280737-ba00-45bb-a8f3-04da514474ac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:47.401539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86280737-ba00-45bb-a8f3-04da514474ac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:47.401971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86280737-ba00-45bb-a8f3-04da514474ac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:47.431720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-86280737-ba00-45bb-a8f3-04da514474ac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049115 May 25 16:21:47.432268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 305/597] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:47 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:47.529231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-64c506ea-8521-4898-92f9-fbe2e8debc54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:47.530168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-64c506ea-8521-4898-92f9-fbe2e8debc54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:47.537096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64c506ea-8521-4898-92f9-fbe2e8debc54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:47.537578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64c506ea-8521-4898-92f9-fbe2e8debc54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:47.538335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64c506ea-8521-4898-92f9-fbe2e8debc54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:47.538766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64c506ea-8521-4898-92f9-fbe2e8debc54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:47.568463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-64c506ea-8521-4898-92f9-fbe2e8debc54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043351 May 25 16:21:47.569021 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 293/598] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:47.659793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d8a75412-d377-4c47-af9c-01c2c5814fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:47.660738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d8a75412-d377-4c47-af9c-01c2c5814fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:47.669627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d8a75412-d377-4c47-af9c-01c2c5814fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:47.670067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d8a75412-d377-4c47-af9c-01c2c5814fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:47.670817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d8a75412-d377-4c47-af9c-01c2c5814fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:47.671279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d8a75412-d377-4c47-af9c-01c2c5814fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:47.821961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d8a75412-d377-4c47-af9c-01c2c5814fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:47.830339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d8a75412-d377-4c47-af9c-01c2c5814fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.177032 May 25 16:21:47.830915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 306/599] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:47 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:48.447588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cf0d6dec-1e48-4e9c-82b8-eb1621746ee6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:48.448649 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cf0d6dec-1e48-4e9c-82b8-eb1621746ee6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:48.455672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf0d6dec-1e48-4e9c-82b8-eb1621746ee6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:48.456166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf0d6dec-1e48-4e9c-82b8-eb1621746ee6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:48.457015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf0d6dec-1e48-4e9c-82b8-eb1621746ee6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:48.457488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf0d6dec-1e48-4e9c-82b8-eb1621746ee6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:48.489394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cf0d6dec-1e48-4e9c-82b8-eb1621746ee6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048158 May 25 16:21:48.489975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 294/600] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:48 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:48.580943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0d074451-81b7-4589-98ec-72c5a0c45757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:48.581907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0d074451-81b7-4589-98ec-72c5a0c45757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:48.588745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d074451-81b7-4589-98ec-72c5a0c45757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:48.589196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d074451-81b7-4589-98ec-72c5a0c45757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:48.589950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d074451-81b7-4589-98ec-72c5a0c45757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:48.590386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d074451-81b7-4589-98ec-72c5a0c45757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:48.619990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0d074451-81b7-4589-98ec-72c5a0c45757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042922 May 25 16:21:48.620434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 307/601] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:48.851544 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7f8ed1d4-900b-452d-bf34-5ac1bf2a51d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:48.852504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7f8ed1d4-900b-452d-bf34-5ac1bf2a51d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:48.863678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f8ed1d4-900b-452d-bf34-5ac1bf2a51d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:48.864426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f8ed1d4-900b-452d-bf34-5ac1bf2a51d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:48.865686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f8ed1d4-900b-452d-bf34-5ac1bf2a51d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:48.866414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f8ed1d4-900b-452d-bf34-5ac1bf2a51d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:49.011896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7f8ed1d4-900b-452d-bf34-5ac1bf2a51d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:49.020581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7f8ed1d4-900b-452d-bf34-5ac1bf2a51d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.175557 May 25 16:21:49.021163 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 295/602] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:48 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:49.506067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e8434070-0380-4f38-ac42-73c7a8a08ca9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:49.507729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e8434070-0380-4f38-ac42-73c7a8a08ca9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:49.518708 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8434070-0380-4f38-ac42-73c7a8a08ca9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:49.519510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8434070-0380-4f38-ac42-73c7a8a08ca9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:49.520771 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8434070-0380-4f38-ac42-73c7a8a08ca9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:49.521485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8434070-0380-4f38-ac42-73c7a8a08ca9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:49.552983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e8434070-0380-4f38-ac42-73c7a8a08ca9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053870 May 25 16:21:49.553565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 308/603] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:49 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:49.636379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-af78688e-b661-4a0e-bfab-696880f686d0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:49.637918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-af78688e-b661-4a0e-bfab-696880f686d0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:49.648429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-af78688e-b661-4a0e-bfab-696880f686d0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:49.649157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-af78688e-b661-4a0e-bfab-696880f686d0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:49.649940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-af78688e-b661-4a0e-bfab-696880f686d0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:49.650382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-af78688e-b661-4a0e-bfab-696880f686d0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:49.680585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-af78688e-b661-4a0e-bfab-696880f686d0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050969 May 25 16:21:49.681140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 296/604] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:50.046128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f1c15b8d-e618-4968-ae6e-d59a26750422 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:50.046742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f1c15b8d-e618-4968-ae6e-d59a26750422 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:50.054134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1c15b8d-e618-4968-ae6e-d59a26750422 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:50.054710 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1c15b8d-e618-4968-ae6e-d59a26750422 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:50.055596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1c15b8d-e618-4968-ae6e-d59a26750422 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:50.056029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1c15b8d-e618-4968-ae6e-d59a26750422 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:50.203178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f1c15b8d-e618-4968-ae6e-d59a26750422 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:50.214139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f1c15b8d-e618-4968-ae6e-d59a26750422 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.171854 May 25 16:21:50.214719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 309/605] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:50 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:50.569713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0d70686f-f7d0-4850-a9c6-2eb5522a59b2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:50.571242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0d70686f-f7d0-4850-a9c6-2eb5522a59b2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:50.578210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d70686f-f7d0-4850-a9c6-2eb5522a59b2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:50.578640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d70686f-f7d0-4850-a9c6-2eb5522a59b2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:50.579429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d70686f-f7d0-4850-a9c6-2eb5522a59b2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:50.579876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0d70686f-f7d0-4850-a9c6-2eb5522a59b2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:50.609630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0d70686f-f7d0-4850-a9c6-2eb5522a59b2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046850 May 25 16:21:50.610181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 297/606] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:50 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:50.695709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a9b05bf1-4158-406f-a9da-a6abbd3a9af7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:50.696598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a9b05bf1-4158-406f-a9da-a6abbd3a9af7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:50.703938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9b05bf1-4158-406f-a9da-a6abbd3a9af7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:50.704388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9b05bf1-4158-406f-a9da-a6abbd3a9af7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:50.705151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9b05bf1-4158-406f-a9da-a6abbd3a9af7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:50.705585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9b05bf1-4158-406f-a9da-a6abbd3a9af7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:50.736674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a9b05bf1-4158-406f-a9da-a6abbd3a9af7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047040 May 25 16:21:50.737221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 310/607] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:50 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:51.236183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b21d50b0-c3e0-4226-abd7-f1fc7c8c5f0a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:51.237192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b21d50b0-c3e0-4226-abd7-f1fc7c8c5f0a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:51.247467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b21d50b0-c3e0-4226-abd7-f1fc7c8c5f0a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:51.248210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b21d50b0-c3e0-4226-abd7-f1fc7c8c5f0a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:51.249457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b21d50b0-c3e0-4226-abd7-f1fc7c8c5f0a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:51.250173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b21d50b0-c3e0-4226-abd7-f1fc7c8c5f0a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:51.410340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b21d50b0-c3e0-4226-abd7-f1fc7c8c5f0a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:51.413623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:51.421382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b21d50b0-c3e0-4226-abd7-f1fc7c8c5f0a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.191692 May 25 16:21:51.421955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 298/608] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:51 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:51.624933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-247cf93d-c321-4be7-8a50-5db971118d81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:51.626306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-247cf93d-c321-4be7-8a50-5db971118d81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:51.636696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-247cf93d-c321-4be7-8a50-5db971118d81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:51.637432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-247cf93d-c321-4be7-8a50-5db971118d81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:51.638630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-247cf93d-c321-4be7-8a50-5db971118d81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:51.639455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-247cf93d-c321-4be7-8a50-5db971118d81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:51.674332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-247cf93d-c321-4be7-8a50-5db971118d81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056123 May 25 16:21:51.674886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 311/609] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:51 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:51.752811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a637dd98-d44c-48e5-a5ea-3db74a1c5031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:51.754299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a637dd98-d44c-48e5-a5ea-3db74a1c5031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:51.765554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a637dd98-d44c-48e5-a5ea-3db74a1c5031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:51.766303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a637dd98-d44c-48e5-a5ea-3db74a1c5031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:51.767617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a637dd98-d44c-48e5-a5ea-3db74a1c5031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:51.768329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a637dd98-d44c-48e5-a5ea-3db74a1c5031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:51.800308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a637dd98-d44c-48e5-a5ea-3db74a1c5031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054179 May 25 16:21:51.800869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 299/610] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:52.438046 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fef2bec7-a124-4726-991f-cc26189344fc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:52.438639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fef2bec7-a124-4726-991f-cc26189344fc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:52.446145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fef2bec7-a124-4726-991f-cc26189344fc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:52.446584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fef2bec7-a124-4726-991f-cc26189344fc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:52.447384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fef2bec7-a124-4726-991f-cc26189344fc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:52.447813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fef2bec7-a124-4726-991f-cc26189344fc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:52.589985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fef2bec7-a124-4726-991f-cc26189344fc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:52.592228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:52.599931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fef2bec7-a124-4726-991f-cc26189344fc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.165768 May 25 16:21:52.600516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 312/611] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:52 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:52.686254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-75f51875-9497-4440-9b2c-38d691009122 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:52.687188 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-75f51875-9497-4440-9b2c-38d691009122 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:52.693552 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75f51875-9497-4440-9b2c-38d691009122 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:52.693995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75f51875-9497-4440-9b2c-38d691009122 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:52.694765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75f51875-9497-4440-9b2c-38d691009122 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:52.695182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75f51875-9497-4440-9b2c-38d691009122 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:52.724405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-75f51875-9497-4440-9b2c-38d691009122 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042069 May 25 16:21:52.724948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 300/612] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:52 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:52.816533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7b2e8e8f-04b1-40c4-87af-70cf58b3cea7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:52.818025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7b2e8e8f-04b1-40c4-87af-70cf58b3cea7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:52.825386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b2e8e8f-04b1-40c4-87af-70cf58b3cea7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:52.825833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b2e8e8f-04b1-40c4-87af-70cf58b3cea7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:52.826597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b2e8e8f-04b1-40c4-87af-70cf58b3cea7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:52.827027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b2e8e8f-04b1-40c4-87af-70cf58b3cea7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:52.856794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7b2e8e8f-04b1-40c4-87af-70cf58b3cea7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046970 May 25 16:21:52.857241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 313/613] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:53.621353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-759038b2-d21d-4529-9df2-b5490e41abb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:53.622360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-759038b2-d21d-4529-9df2-b5490e41abb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:53.633373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-759038b2-d21d-4529-9df2-b5490e41abb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:53.633835 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-759038b2-d21d-4529-9df2-b5490e41abb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:53.634617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-759038b2-d21d-4529-9df2-b5490e41abb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:53.635029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-759038b2-d21d-4529-9df2-b5490e41abb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:53.738713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-94031bab-7937-4070-97b2-180673d474f7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:53.740405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-94031bab-7937-4070-97b2-180673d474f7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:53.752140 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94031bab-7937-4070-97b2-180673d474f7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:53.752751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94031bab-7937-4070-97b2-180673d474f7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:53.754006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94031bab-7937-4070-97b2-180673d474f7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:53.754784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94031bab-7937-4070-97b2-180673d474f7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:53.793484 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-94031bab-7937-4070-97b2-180673d474f7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061052 May 25 16:21:53.794061 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 314/614] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:53 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:53.870459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0b6c89b4-3892-424b-95dd-438849f11d19 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:53.871423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0b6c89b4-3892-424b-95dd-438849f11d19 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:53.878292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b6c89b4-3892-424b-95dd-438849f11d19 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:53.878750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b6c89b4-3892-424b-95dd-438849f11d19 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:53.879569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b6c89b4-3892-424b-95dd-438849f11d19 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:53.880009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b6c89b4-3892-424b-95dd-438849f11d19 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:53.913601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0b6c89b4-3892-424b-95dd-438849f11d19 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047193 May 25 16:21:53.914230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 315/615] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:53 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:54.033832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-759038b2-d21d-4529-9df2-b5490e41abb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:54.044412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-759038b2-d21d-4529-9df2-b5490e41abb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.429456 May 25 16:21:54.044927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 301/616] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:53 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:54.808114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-24bfde1c-7a02-4432-81c7-337ba895067e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:54.809042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-24bfde1c-7a02-4432-81c7-337ba895067e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:54.815935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-24bfde1c-7a02-4432-81c7-337ba895067e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:54.816473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-24bfde1c-7a02-4432-81c7-337ba895067e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:54.817228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-24bfde1c-7a02-4432-81c7-337ba895067e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:54.817653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-24bfde1c-7a02-4432-81c7-337ba895067e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:54.847324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-24bfde1c-7a02-4432-81c7-337ba895067e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044331 May 25 16:21:54.847988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 316/617] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:54 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:54.930297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-87775d8b-2d3f-44dc-a03f-f025dd1682f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:54.931811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-87775d8b-2d3f-44dc-a03f-f025dd1682f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:54.935434 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=80 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:54.944373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87775d8b-2d3f-44dc-a03f-f025dd1682f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:54.945082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87775d8b-2d3f-44dc-a03f-f025dd1682f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:54.946907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87775d8b-2d3f-44dc-a03f-f025dd1682f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:54.948033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87775d8b-2d3f-44dc-a03f-f025dd1682f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:54.978716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-87775d8b-2d3f-44dc-a03f-f025dd1682f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055069 May 25 16:21:54.979464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 302/618] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:55.065771 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f1ecf5f6-a64d-4cbd-93e1-ac850c2a35f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:55.066831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f1ecf5f6-a64d-4cbd-93e1-ac850c2a35f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:55.074829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1ecf5f6-a64d-4cbd-93e1-ac850c2a35f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:55.075269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1ecf5f6-a64d-4cbd-93e1-ac850c2a35f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:55.076087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1ecf5f6-a64d-4cbd-93e1-ac850c2a35f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:55.076515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1ecf5f6-a64d-4cbd-93e1-ac850c2a35f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:55.222681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f1ecf5f6-a64d-4cbd-93e1-ac850c2a35f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:55.232440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f1ecf5f6-a64d-4cbd-93e1-ac850c2a35f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.173216 May 25 16:21:55.232991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 317/619] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:55 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:55.859429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bc5b6725-8ed7-4b48-919d-10cab208f726 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:55.860363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bc5b6725-8ed7-4b48-919d-10cab208f726 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:55.866915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc5b6725-8ed7-4b48-919d-10cab208f726 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:55.867405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc5b6725-8ed7-4b48-919d-10cab208f726 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:55.868171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc5b6725-8ed7-4b48-919d-10cab208f726 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:55.868728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc5b6725-8ed7-4b48-919d-10cab208f726 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:55.898405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bc5b6725-8ed7-4b48-919d-10cab208f726 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042975 May 25 16:21:55.898950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 303/620] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:55 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:55.994852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ca2370f3-8a55-4126-ad78-df0c8e09b3e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:55.996343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ca2370f3-8a55-4126-ad78-df0c8e09b3e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:56.007469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca2370f3-8a55-4126-ad78-df0c8e09b3e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:56.008213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca2370f3-8a55-4126-ad78-df0c8e09b3e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:56.008962 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca2370f3-8a55-4126-ad78-df0c8e09b3e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:56.009393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca2370f3-8a55-4126-ad78-df0c8e09b3e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:56.038967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ca2370f3-8a55-4126-ad78-df0c8e09b3e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050911 May 25 16:21:56.039566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 318/621] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:55 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:56.254533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f831a4f0-d930-4817-828c-eaff9fce29ed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:56.255412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f831a4f0-d930-4817-828c-eaff9fce29ed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:56.266330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f831a4f0-d930-4817-828c-eaff9fce29ed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:56.267081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f831a4f0-d930-4817-828c-eaff9fce29ed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:56.268397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f831a4f0-d930-4817-828c-eaff9fce29ed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:56.269122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f831a4f0-d930-4817-828c-eaff9fce29ed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:56.417471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f831a4f0-d930-4817-828c-eaff9fce29ed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:56.426034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f831a4f0-d930-4817-828c-eaff9fce29ed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.178206 May 25 16:21:56.426593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 304/622] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:56 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:56.909939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9ab92887-cb28-4de6-9145-0f553ebade6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:56.910867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9ab92887-cb28-4de6-9145-0f553ebade6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:56.917862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ab92887-cb28-4de6-9145-0f553ebade6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:56.918318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ab92887-cb28-4de6-9145-0f553ebade6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:56.919083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ab92887-cb28-4de6-9145-0f553ebade6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:56.919550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ab92887-cb28-4de6-9145-0f553ebade6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:56.951187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9ab92887-cb28-4de6-9145-0f553ebade6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045239 May 25 16:21:56.951769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 319/623] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:56 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:57.055557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-520f3f56-1465-4f58-9746-4850fc4ebeb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:57.056688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-520f3f56-1465-4f58-9746-4850fc4ebeb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:57.064129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-520f3f56-1465-4f58-9746-4850fc4ebeb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:57.064656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-520f3f56-1465-4f58-9746-4850fc4ebeb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:57.065587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-520f3f56-1465-4f58-9746-4850fc4ebeb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:57.066094 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-520f3f56-1465-4f58-9746-4850fc4ebeb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:57.098901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-520f3f56-1465-4f58-9746-4850fc4ebeb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049917 May 25 16:21:57.099499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 305/624] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:57.444696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-75ad0441-089b-45c1-b708-359404419f43 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:57.445068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-75ad0441-089b-45c1-b708-359404419f43 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:57.452546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-75ad0441-089b-45c1-b708-359404419f43 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:57.452999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-75ad0441-089b-45c1-b708-359404419f43 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:57.453769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-75ad0441-089b-45c1-b708-359404419f43 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:57.454390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-75ad0441-089b-45c1-b708-359404419f43 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:57.598052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-75ad0441-089b-45c1-b708-359404419f43 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:57.606286 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-75ad0441-089b-45c1-b708-359404419f43 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.166460 May 25 16:21:57.606856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 320/625] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:57 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:57.966574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3c75bc54-b2a6-4eaa-9db4-217c1676095b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:57.968225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3c75bc54-b2a6-4eaa-9db4-217c1676095b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:57.978390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3c75bc54-b2a6-4eaa-9db4-217c1676095b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:57.978828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3c75bc54-b2a6-4eaa-9db4-217c1676095b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:57.979603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3c75bc54-b2a6-4eaa-9db4-217c1676095b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:57.980028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3c75bc54-b2a6-4eaa-9db4-217c1676095b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.009681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3c75bc54-b2a6-4eaa-9db4-217c1676095b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050156 May 25 16:21:58.010231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 306/626] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:57 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:58.111678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dded759e-92a4-4dee-87fd-0528b802d42f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:58.112643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dded759e-92a4-4dee-87fd-0528b802d42f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:58.119509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dded759e-92a4-4dee-87fd-0528b802d42f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.119971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dded759e-92a4-4dee-87fd-0528b802d42f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.120730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dded759e-92a4-4dee-87fd-0528b802d42f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.121164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dded759e-92a4-4dee-87fd-0528b802d42f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.152405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dded759e-92a4-4dee-87fd-0528b802d42f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044783 May 25 16:21:58.152953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 321/627] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:58 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:58.307555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c914b0a8-2afc-4f16-8332-0d75a6e6c09a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:58.308489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c914b0a8-2afc-4f16-8332-0d75a6e6c09a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:58.314844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c914b0a8-2afc-4f16-8332-0d75a6e6c09a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.315334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c914b0a8-2afc-4f16-8332-0d75a6e6c09a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.316116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c914b0a8-2afc-4f16-8332-0d75a6e6c09a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.316526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c914b0a8-2afc-4f16-8332-0d75a6e6c09a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.348623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c914b0a8-2afc-4f16-8332-0d75a6e6c09a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046286 May 25 16:21:58.349400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 307/628] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:58 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 25 16:21:58.409237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-544779f9-2ce5-4cc7-88ef-8650bef945f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:58.409757 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-544779f9-2ce5-4cc7-88ef-8650bef945f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:58.411596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=87,nova_api:DELETE=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:58.417495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-544779f9-2ce5-4cc7-88ef-8650bef945f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.417945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-544779f9-2ce5-4cc7-88ef-8650bef945f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.418707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-544779f9-2ce5-4cc7-88ef-8650bef945f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.419147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-544779f9-2ce5-4cc7-88ef-8650bef945f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.444756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-544779f9-2ce5-4cc7-88ef-8650bef945f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:21:58.529932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-544779f9-2ce5-4cc7-88ef-8650bef945f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "DELETE /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 204 len: 0 microversion: 2.1 time: 0.124311 May 25 16:21:58.530545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 322/629] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:21:58 2022] DELETE /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:21:58.541775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9125fcba-72b5-4df3-8997-5f4eed28f1bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:58.542676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9125fcba-72b5-4df3-8997-5f4eed28f1bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:58.548991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9125fcba-72b5-4df3-8997-5f4eed28f1bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.549457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9125fcba-72b5-4df3-8997-5f4eed28f1bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.550141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9125fcba-72b5-4df3-8997-5f4eed28f1bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.550572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9125fcba-72b5-4df3-8997-5f4eed28f1bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.628438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-69b69559-17c4-4c93-8d19-072fea5adc5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:58.629167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-69b69559-17c4-4c93-8d19-072fea5adc5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:58.635479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69b69559-17c4-4c93-8d19-072fea5adc5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.635930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69b69559-17c4-4c93-8d19-072fea5adc5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.636693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69b69559-17c4-4c93-8d19-072fea5adc5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:58.637126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69b69559-17c4-4c93-8d19-072fea5adc5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:58.690968 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9125fcba-72b5-4df3-8997-5f4eed28f1bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:58.698995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9125fcba-72b5-4df3-8997-5f4eed28f1bb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.163314 May 25 16:21:58.699607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 308/630] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:58 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:58.790725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-69b69559-17c4-4c93-8d19-072fea5adc5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:58.799187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-69b69559-17c4-4c93-8d19-072fea5adc5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.175020 May 25 16:21:58.799809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 323/631] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:58 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:59.021926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bfae462f-b2f2-4b1b-ae88-4df80a71d190 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:59.022863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bfae462f-b2f2-4b1b-ae88-4df80a71d190 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:59.029877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bfae462f-b2f2-4b1b-ae88-4df80a71d190 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:59.030329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bfae462f-b2f2-4b1b-ae88-4df80a71d190 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:59.031131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bfae462f-b2f2-4b1b-ae88-4df80a71d190 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:59.031601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bfae462f-b2f2-4b1b-ae88-4df80a71d190 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:59.061914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bfae462f-b2f2-4b1b-ae88-4df80a71d190 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043940 May 25 16:21:59.062403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 309/632] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:21:59 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:21:59.166616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9e08ceaf-4e57-4313-9139-432e8f1fd62b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:59.167689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9e08ceaf-4e57-4313-9139-432e8f1fd62b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:59.175562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e08ceaf-4e57-4313-9139-432e8f1fd62b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:59.176069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e08ceaf-4e57-4313-9139-432e8f1fd62b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:59.176905 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e08ceaf-4e57-4313-9139-432e8f1fd62b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:59.177385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e08ceaf-4e57-4313-9139-432e8f1fd62b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:59.209066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9e08ceaf-4e57-4313-9139-432e8f1fd62b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047442 May 25 16:21:59.209625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 324/633] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:21:59 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:21:59.720211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-22b12b2a-d374-4337-954e-d9950fab1178 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:59.721203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-22b12b2a-d374-4337-954e-d9950fab1178 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:59.729781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22b12b2a-d374-4337-954e-d9950fab1178 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:59.730223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22b12b2a-d374-4337-954e-d9950fab1178 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:59.730972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22b12b2a-d374-4337-954e-d9950fab1178 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:59.731511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22b12b2a-d374-4337-954e-d9950fab1178 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:59.817664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-79946e08-f849-4709-a71e-dcd17111d6f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:21:59.818268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-79946e08-f849-4709-a71e-dcd17111d6f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:21:59.824475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-79946e08-f849-4709-a71e-dcd17111d6f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:59.824925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-79946e08-f849-4709-a71e-dcd17111d6f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:59.825684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-79946e08-f849-4709-a71e-dcd17111d6f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:21:59.826116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-79946e08-f849-4709-a71e-dcd17111d6f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:21:59.862957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-22b12b2a-d374-4337-954e-d9950fab1178 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:59.872053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-22b12b2a-d374-4337-954e-d9950fab1178 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.158062 May 25 16:21:59.872650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 310/634] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:59 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:21:59.982924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-79946e08-f849-4709-a71e-dcd17111d6f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:21:59.991831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-79946e08-f849-4709-a71e-dcd17111d6f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.177958 May 25 16:21:59.992524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 325/635] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:21:59 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:00.074922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c80f8f18-4825-463a-a7fe-5354ef6c882b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:00.076441 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c80f8f18-4825-463a-a7fe-5354ef6c882b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:00.086169 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c80f8f18-4825-463a-a7fe-5354ef6c882b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:00.086633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c80f8f18-4825-463a-a7fe-5354ef6c882b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:00.087424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c80f8f18-4825-463a-a7fe-5354ef6c882b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:00.087854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c80f8f18-4825-463a-a7fe-5354ef6c882b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:00.117661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c80f8f18-4825-463a-a7fe-5354ef6c882b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048412 May 25 16:22:00.118220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 311/636] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:00 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:00.225038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cbffdd53-4951-4db3-a7c3-33ebca16517d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:00.226722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cbffdd53-4951-4db3-a7c3-33ebca16517d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:00.233997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cbffdd53-4951-4db3-a7c3-33ebca16517d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:00.234476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cbffdd53-4951-4db3-a7c3-33ebca16517d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:00.235335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cbffdd53-4951-4db3-a7c3-33ebca16517d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:00.235801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cbffdd53-4951-4db3-a7c3-33ebca16517d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:00.266909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cbffdd53-4951-4db3-a7c3-33ebca16517d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048645 May 25 16:22:00.267502 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 326/637] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:00 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:00.894974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f0245e2b-b4e3-468e-88f7-9b9e282e6b91 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:00.895888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f0245e2b-b4e3-468e-88f7-9b9e282e6b91 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:00.903611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f0245e2b-b4e3-468e-88f7-9b9e282e6b91 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:00.904069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f0245e2b-b4e3-468e-88f7-9b9e282e6b91 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:00.904823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f0245e2b-b4e3-468e-88f7-9b9e282e6b91 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:00.905255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f0245e2b-b4e3-468e-88f7-9b9e282e6b91 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:01.011980 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-32d1a64d-e0dc-479e-a534-9447617d6f5d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:01.012672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-32d1a64d-e0dc-479e-a534-9447617d6f5d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:01.019724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32d1a64d-e0dc-479e-a534-9447617d6f5d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:01.020214 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32d1a64d-e0dc-479e-a534-9447617d6f5d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:01.021062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32d1a64d-e0dc-479e-a534-9447617d6f5d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:01.021562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32d1a64d-e0dc-479e-a534-9447617d6f5d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:01.042736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f0245e2b-b4e3-468e-88f7-9b9e282e6b91 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:01.050421 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f0245e2b-b4e3-468e-88f7-9b9e282e6b91 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.162610 May 25 16:22:01.051036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 312/638] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:00 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:01.130763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f86f6cbe-7c83-4711-864e-e9769c7e9a4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:01.131848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f86f6cbe-7c83-4711-864e-e9769c7e9a4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:01.138843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f86f6cbe-7c83-4711-864e-e9769c7e9a4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:01.139320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f86f6cbe-7c83-4711-864e-e9769c7e9a4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:01.140077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f86f6cbe-7c83-4711-864e-e9769c7e9a4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:01.140504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f86f6cbe-7c83-4711-864e-e9769c7e9a4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:01.171476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f86f6cbe-7c83-4711-864e-e9769c7e9a4a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044498 May 25 16:22:01.171930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 313/639] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:01 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:22:01.177316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-32d1a64d-e0dc-479e-a534-9447617d6f5d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:01.179628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:01.187323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-32d1a64d-e0dc-479e-a534-9447617d6f5d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.179803 May 25 16:22:01.187895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 327/640] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:01 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:01.283138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-929322e3-65b9-4faa-86bb-d683c40883a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:01.284710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-929322e3-65b9-4faa-86bb-d683c40883a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:01.295409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-929322e3-65b9-4faa-86bb-d683c40883a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:01.296106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-929322e3-65b9-4faa-86bb-d683c40883a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:01.297360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-929322e3-65b9-4faa-86bb-d683c40883a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:01.298079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-929322e3-65b9-4faa-86bb-d683c40883a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:01.330773 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-929322e3-65b9-4faa-86bb-d683c40883a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053162 May 25 16:22:01.331371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 314/641] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:01 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:02.072368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3e224e8d-f219-44b1-8c98-d8e976d8d18c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:02.073413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3e224e8d-f219-44b1-8c98-d8e976d8d18c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:02.085716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e224e8d-f219-44b1-8c98-d8e976d8d18c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:02.086399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e224e8d-f219-44b1-8c98-d8e976d8d18c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:02.087187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e224e8d-f219-44b1-8c98-d8e976d8d18c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:02.087649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e224e8d-f219-44b1-8c98-d8e976d8d18c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:02.184259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-84d6437b-9804-4682-ba33-a0ca03dadf50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:02.185207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-84d6437b-9804-4682-ba33-a0ca03dadf50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:02.192185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84d6437b-9804-4682-ba33-a0ca03dadf50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:02.192661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84d6437b-9804-4682-ba33-a0ca03dadf50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:02.193411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84d6437b-9804-4682-ba33-a0ca03dadf50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:02.193863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84d6437b-9804-4682-ba33-a0ca03dadf50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:02.208241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:02.224426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-84d6437b-9804-4682-ba33-a0ca03dadf50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044536 May 25 16:22:02.224966 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 315/642] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:02 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:02.228752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3e224e8d-f219-44b1-8c98-d8e976d8d18c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:02.231007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-34db4ef5-339d-4876-a3d7-b43af21eeb49 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:02.231622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-34db4ef5-339d-4876-a3d7-b43af21eeb49 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:02.236485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3e224e8d-f219-44b1-8c98-d8e976d8d18c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.171174 May 25 16:22:02.237049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 328/643] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:02 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:02.237938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34db4ef5-339d-4876-a3d7-b43af21eeb49 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:02.238382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34db4ef5-339d-4876-a3d7-b43af21eeb49 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:02.239137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34db4ef5-339d-4876-a3d7-b43af21eeb49 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:02.239594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34db4ef5-339d-4876-a3d7-b43af21eeb49 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:02.342721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dae43b51-6d3d-4c9c-8739-12c28dd35fe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:02.343670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dae43b51-6d3d-4c9c-8739-12c28dd35fe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:02.349991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dae43b51-6d3d-4c9c-8739-12c28dd35fe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:02.350453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dae43b51-6d3d-4c9c-8739-12c28dd35fe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:02.351228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dae43b51-6d3d-4c9c-8739-12c28dd35fe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:02.351798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dae43b51-6d3d-4c9c-8739-12c28dd35fe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:02.383470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dae43b51-6d3d-4c9c-8739-12c28dd35fe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044740 May 25 16:22:02.384060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 329/644] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:02 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:02.386037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-34db4ef5-339d-4876-a3d7-b43af21eeb49 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:02.394394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-34db4ef5-339d-4876-a3d7-b43af21eeb49 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.168468 May 25 16:22:02.395000 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 316/645] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:02 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:03.237237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4c537cde-b239-46e1-94ab-7ccc0445ce14 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:03.238207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4c537cde-b239-46e1-94ab-7ccc0445ce14 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:03.245747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c537cde-b239-46e1-94ab-7ccc0445ce14 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:03.246234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c537cde-b239-46e1-94ab-7ccc0445ce14 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:03.247016 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c537cde-b239-46e1-94ab-7ccc0445ce14 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:03.247520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c537cde-b239-46e1-94ab-7ccc0445ce14 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:03.254153 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2b855aaa-55b1-4e04-b454-beecf4fc1be1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:03.254897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2b855aaa-55b1-4e04-b454-beecf4fc1be1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:03.261996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b855aaa-55b1-4e04-b454-beecf4fc1be1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:03.262445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b855aaa-55b1-4e04-b454-beecf4fc1be1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:03.263281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b855aaa-55b1-4e04-b454-beecf4fc1be1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:03.263717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b855aaa-55b1-4e04-b454-beecf4fc1be1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:03.277996 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4c537cde-b239-46e1-94ab-7ccc0445ce14 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044928 May 25 16:22:03.278547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 330/646] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:03 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:03.396197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7ca0f296-3859-48ef-870f-8126c1719fd1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:03.397115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7ca0f296-3859-48ef-870f-8126c1719fd1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:03.403499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca0f296-3859-48ef-870f-8126c1719fd1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:03.403945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca0f296-3859-48ef-870f-8126c1719fd1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:03.404706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca0f296-3859-48ef-870f-8126c1719fd1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:03.405130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca0f296-3859-48ef-870f-8126c1719fd1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:03.409237 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2b855aaa-55b1-4e04-b454-beecf4fc1be1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:03.417464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2b855aaa-55b1-4e04-b454-beecf4fc1be1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.167321 May 25 16:22:03.418107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 317/647] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:03 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:03.422836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d13d45dd-740b-44e5-b5bd-78aa504982d2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:03.423591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d13d45dd-740b-44e5-b5bd-78aa504982d2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:03.429652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d13d45dd-740b-44e5-b5bd-78aa504982d2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:03.430128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d13d45dd-740b-44e5-b5bd-78aa504982d2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:03.430892 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d13d45dd-740b-44e5-b5bd-78aa504982d2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:03.431365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d13d45dd-740b-44e5-b5bd-78aa504982d2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:03.436538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7ca0f296-3859-48ef-870f-8126c1719fd1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044373 May 25 16:22:03.437140 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 331/648] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:03 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:03.575380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d13d45dd-740b-44e5-b5bd-78aa504982d2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:03.584118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d13d45dd-740b-44e5-b5bd-78aa504982d2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.165031 May 25 16:22:03.584688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 318/649] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:03 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:04.294450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d67b0dd8-1e2d-4071-9bb6-212d9459d01d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:04.295975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d67b0dd8-1e2d-4071-9bb6-212d9459d01d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:04.306185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d67b0dd8-1e2d-4071-9bb6-212d9459d01d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:04.306622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d67b0dd8-1e2d-4071-9bb6-212d9459d01d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:04.307408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d67b0dd8-1e2d-4071-9bb6-212d9459d01d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:04.307832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d67b0dd8-1e2d-4071-9bb6-212d9459d01d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:04.338034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d67b0dd8-1e2d-4071-9bb6-212d9459d01d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050706 May 25 16:22:04.338506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 332/650] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:04 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:04.437866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-70482521-feb4-44e3-a158-779f6c8ead5e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:04.438891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-70482521-feb4-44e3-a158-779f6c8ead5e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:04.450165 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-70482521-feb4-44e3-a158-779f6c8ead5e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:04.450956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-70482521-feb4-44e3-a158-779f6c8ead5e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:04.451949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8b296e37-42eb-45f2-8813-f91d90f748e1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:04.452184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-70482521-feb4-44e3-a158-779f6c8ead5e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:04.452959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-70482521-feb4-44e3-a158-779f6c8ead5e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:04.453451 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8b296e37-42eb-45f2-8813-f91d90f748e1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:04.462942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8b296e37-42eb-45f2-8813-f91d90f748e1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:04.463453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8b296e37-42eb-45f2-8813-f91d90f748e1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:04.464193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8b296e37-42eb-45f2-8813-f91d90f748e1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:04.464658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8b296e37-42eb-45f2-8813-f91d90f748e1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:04.494827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8b296e37-42eb-45f2-8813-f91d90f748e1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049853 May 25 16:22:04.495649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 333/651] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:04 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:04.592625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-70482521-feb4-44e3-a158-779f6c8ead5e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:04.601899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-70482521-feb4-44e3-a158-779f6c8ead5e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.170223 May 25 16:22:04.602127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-912f3c27-fa80-45e1-acd2-522ce19f9240 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:04.602633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 319/652] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:04 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:04.602791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-912f3c27-fa80-45e1-acd2-522ce19f9240 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:04.609024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-912f3c27-fa80-45e1-acd2-522ce19f9240 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:04.609476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-912f3c27-fa80-45e1-acd2-522ce19f9240 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:04.610237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-912f3c27-fa80-45e1-acd2-522ce19f9240 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:04.610675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-912f3c27-fa80-45e1-acd2-522ce19f9240 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:04.754474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-912f3c27-fa80-45e1-acd2-522ce19f9240 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:04.763047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-912f3c27-fa80-45e1-acd2-522ce19f9240 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.165004 May 25 16:22:04.763655 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 334/653] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:04 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:05.354882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cf4b036f-16fc-404b-b3bd-3ed4fd15398d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:05.356450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cf4b036f-16fc-404b-b3bd-3ed4fd15398d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:05.366532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf4b036f-16fc-404b-b3bd-3ed4fd15398d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:05.367386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf4b036f-16fc-404b-b3bd-3ed4fd15398d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:05.368201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf4b036f-16fc-404b-b3bd-3ed4fd15398d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:05.368631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf4b036f-16fc-404b-b3bd-3ed4fd15398d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:05.400730 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cf4b036f-16fc-404b-b3bd-3ed4fd15398d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052505 May 25 16:22:05.401294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 320/654] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:05 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:05.509890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f822ddfa-7415-47c1-b3d0-cb50e3582cc3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:05.511028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f822ddfa-7415-47c1-b3d0-cb50e3582cc3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:05.519444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f822ddfa-7415-47c1-b3d0-cb50e3582cc3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:05.519999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f822ddfa-7415-47c1-b3d0-cb50e3582cc3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:05.520981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f822ddfa-7415-47c1-b3d0-cb50e3582cc3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:05.521442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f822ddfa-7415-47c1-b3d0-cb50e3582cc3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:05.554962 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f822ddfa-7415-47c1-b3d0-cb50e3582cc3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051309 May 25 16:22:05.555544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 335/655] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:05 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:05.622992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7566defb-b04b-402f-960d-436ad119be06 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:05.623922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7566defb-b04b-402f-960d-436ad119be06 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:05.634164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7566defb-b04b-402f-960d-436ad119be06 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:05.634665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7566defb-b04b-402f-960d-436ad119be06 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:05.635450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7566defb-b04b-402f-960d-436ad119be06 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:05.635877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7566defb-b04b-402f-960d-436ad119be06 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:05.778859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7566defb-b04b-402f-960d-436ad119be06 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:05.780613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-eb8c5f2a-6fac-4a35-9fa5-04ff71db6b0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:05.781444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-eb8c5f2a-6fac-4a35-9fa5-04ff71db6b0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:05.786962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7566defb-b04b-402f-960d-436ad119be06 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.170776 May 25 16:22:05.792370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb8c5f2a-6fac-4a35-9fa5-04ff71db6b0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:05.792559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 321/656] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:05 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:05.792679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb8c5f2a-6fac-4a35-9fa5-04ff71db6b0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:05.792679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb8c5f2a-6fac-4a35-9fa5-04ff71db6b0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:05.792679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb8c5f2a-6fac-4a35-9fa5-04ff71db6b0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:05.933818 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eb8c5f2a-6fac-4a35-9fa5-04ff71db6b0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:05.944091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-eb8c5f2a-6fac-4a35-9fa5-04ff71db6b0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.167260 May 25 16:22:05.944660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 336/657] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:05 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:06.415800 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ac932db2-0b33-4c06-a4ff-3e07fd3e6d0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:06.416830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ac932db2-0b33-4c06-a4ff-3e07fd3e6d0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:06.424059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac932db2-0b33-4c06-a4ff-3e07fd3e6d0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:06.424565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac932db2-0b33-4c06-a4ff-3e07fd3e6d0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:06.425406 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac932db2-0b33-4c06-a4ff-3e07fd3e6d0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:06.425890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac932db2-0b33-4c06-a4ff-3e07fd3e6d0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:06.457254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ac932db2-0b33-4c06-a4ff-3e07fd3e6d0e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048081 May 25 16:22:06.457807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 322/658] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:06 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:06.570345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c46e3fd2-09ed-4472-b834-e0e7c5fea956 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:06.571912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c46e3fd2-09ed-4472-b834-e0e7c5fea956 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:06.582830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c46e3fd2-09ed-4472-b834-e0e7c5fea956 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:06.583632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c46e3fd2-09ed-4472-b834-e0e7c5fea956 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:06.584881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c46e3fd2-09ed-4472-b834-e0e7c5fea956 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:06.585595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c46e3fd2-09ed-4472-b834-e0e7c5fea956 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:06.615749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c46e3fd2-09ed-4472-b834-e0e7c5fea956 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052006 May 25 16:22:06.616305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 337/659] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:06 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:06.806876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cf3caa11-e591-4587-b896-ff89453c7303 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:06.808131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cf3caa11-e591-4587-b896-ff89453c7303 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:06.816606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf3caa11-e591-4587-b896-ff89453c7303 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:06.817205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf3caa11-e591-4587-b896-ff89453c7303 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:06.818223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf3caa11-e591-4587-b896-ff89453c7303 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:06.818807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf3caa11-e591-4587-b896-ff89453c7303 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:06.961502 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0f935bfb-d198-45a9-9636-81d6d86bbd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:06.962112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0f935bfb-d198-45a9-9636-81d6d86bbd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:06.962289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cf3caa11-e591-4587-b896-ff89453c7303 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:06.968405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0f935bfb-d198-45a9-9636-81d6d86bbd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:06.968884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0f935bfb-d198-45a9-9636-81d6d86bbd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:06.969646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0f935bfb-d198-45a9-9636-81d6d86bbd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:06.970077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0f935bfb-d198-45a9-9636-81d6d86bbd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:06.974989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cf3caa11-e591-4587-b896-ff89453c7303 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.173284 May 25 16:22:06.985428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 323/660] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:06 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:07.116855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0f935bfb-d198-45a9-9636-81d6d86bbd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:07.129250 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0f935bfb-d198-45a9-9636-81d6d86bbd79 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.171329 May 25 16:22:07.130003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 338/661] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:06 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:07.474724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-62b7bfba-7aba-4db5-84e3-8b87aa08579f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:07.476121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-62b7bfba-7aba-4db5-84e3-8b87aa08579f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:07.486684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62b7bfba-7aba-4db5-84e3-8b87aa08579f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:07.487493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62b7bfba-7aba-4db5-84e3-8b87aa08579f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:07.488742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62b7bfba-7aba-4db5-84e3-8b87aa08579f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:07.489446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62b7bfba-7aba-4db5-84e3-8b87aa08579f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:07.521728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-62b7bfba-7aba-4db5-84e3-8b87aa08579f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055035 May 25 16:22:07.522296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 324/662] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:07 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:07.631762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e5b15946-e4a3-4097-b21b-ec677b2aa05e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:07.633240 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e5b15946-e4a3-4097-b21b-ec677b2aa05e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:07.644360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5b15946-e4a3-4097-b21b-ec677b2aa05e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:07.645101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5b15946-e4a3-4097-b21b-ec677b2aa05e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:07.646363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5b15946-e4a3-4097-b21b-ec677b2aa05e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:07.647028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5b15946-e4a3-4097-b21b-ec677b2aa05e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:07.678188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e5b15946-e4a3-4097-b21b-ec677b2aa05e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053142 May 25 16:22:07.678747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 339/663] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:07 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:07.997093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-df34aa95-ba57-41a8-b02e-08ca766e7946 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:07.998135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-df34aa95-ba57-41a8-b02e-08ca766e7946 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:08.008739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df34aa95-ba57-41a8-b02e-08ca766e7946 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:08.009195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df34aa95-ba57-41a8-b02e-08ca766e7946 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:08.009951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df34aa95-ba57-41a8-b02e-08ca766e7946 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:08.010387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df34aa95-ba57-41a8-b02e-08ca766e7946 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:08.147050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-43deb8cd-b15c-468d-9390-23e44864ad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:08.147693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-43deb8cd-b15c-468d-9390-23e44864ad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:08.151820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-df34aa95-ba57-41a8-b02e-08ca766e7946 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:08.154365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43deb8cd-b15c-468d-9390-23e44864ad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:08.154811 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43deb8cd-b15c-468d-9390-23e44864ad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:08.155597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43deb8cd-b15c-468d-9390-23e44864ad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:08.156024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43deb8cd-b15c-468d-9390-23e44864ad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:08.159499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-df34aa95-ba57-41a8-b02e-08ca766e7946 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.169002 May 25 16:22:08.160339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 325/664] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:07 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:08.304816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-43deb8cd-b15c-468d-9390-23e44864ad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:08.312882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-43deb8cd-b15c-468d-9390-23e44864ad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.169733 May 25 16:22:08.313448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 340/665] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:08 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:08.537856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5e12c6f2-7d47-423a-93e8-53a5dbe831c0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:08.539329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5e12c6f2-7d47-423a-93e8-53a5dbe831c0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:08.546671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5e12c6f2-7d47-423a-93e8-53a5dbe831c0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:08.547196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5e12c6f2-7d47-423a-93e8-53a5dbe831c0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:08.548156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5e12c6f2-7d47-423a-93e8-53a5dbe831c0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:08.548670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5e12c6f2-7d47-423a-93e8-53a5dbe831c0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:08.581133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5e12c6f2-7d47-423a-93e8-53a5dbe831c0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050343 May 25 16:22:08.581724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 326/666] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:08 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:08.694558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c035aead-7ddd-455b-9302-5d69d8288711 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:08.695817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c035aead-7ddd-455b-9302-5d69d8288711 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:08.703437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c035aead-7ddd-455b-9302-5d69d8288711 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:08.703938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c035aead-7ddd-455b-9302-5d69d8288711 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:08.704778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c035aead-7ddd-455b-9302-5d69d8288711 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:08.705257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c035aead-7ddd-455b-9302-5d69d8288711 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:08.737000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c035aead-7ddd-455b-9302-5d69d8288711 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049231 May 25 16:22:08.737545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 341/667] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:08 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:09.181269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-239b3131-c757-4542-a1ea-e190a0f30129 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:09.182256 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-239b3131-c757-4542-a1ea-e190a0f30129 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:09.193233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-239b3131-c757-4542-a1ea-e190a0f30129 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:09.193978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-239b3131-c757-4542-a1ea-e190a0f30129 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:09.195331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-239b3131-c757-4542-a1ea-e190a0f30129 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:09.195767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-239b3131-c757-4542-a1ea-e190a0f30129 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:09.331268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-35836801-dad4-4c99-ab76-4f114859008a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:09.331551 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-35836801-dad4-4c99-ab76-4f114859008a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:09.333599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-239b3131-c757-4542-a1ea-e190a0f30129 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:09.338908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35836801-dad4-4c99-ab76-4f114859008a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:09.339362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35836801-dad4-4c99-ab76-4f114859008a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:09.340158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35836801-dad4-4c99-ab76-4f114859008a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:09.340583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35836801-dad4-4c99-ab76-4f114859008a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:09.341449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-239b3131-c757-4542-a1ea-e190a0f30129 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.166775 May 25 16:22:09.342029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 327/668] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:09 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:09.499135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-35836801-dad4-4c99-ab76-4f114859008a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:09.510371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-35836801-dad4-4c99-ab76-4f114859008a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.183440 May 25 16:22:09.510970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 342/669] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:09 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:09.597241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6e183fa0-e63f-42d8-9af7-581ecdf7f90e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:09.598615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6e183fa0-e63f-42d8-9af7-581ecdf7f90e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:09.609240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e183fa0-e63f-42d8-9af7-581ecdf7f90e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:09.609981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e183fa0-e63f-42d8-9af7-581ecdf7f90e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:09.611227 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e183fa0-e63f-42d8-9af7-581ecdf7f90e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:09.611953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e183fa0-e63f-42d8-9af7-581ecdf7f90e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:09.643325 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6e183fa0-e63f-42d8-9af7-581ecdf7f90e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052843 May 25 16:22:09.643859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 328/670] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:09 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:09.750312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9ab30c6e-3019-4303-8790-0973c2155718 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:09.751390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9ab30c6e-3019-4303-8790-0973c2155718 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:09.758954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ab30c6e-3019-4303-8790-0973c2155718 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:09.759499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ab30c6e-3019-4303-8790-0973c2155718 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:09.760422 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ab30c6e-3019-4303-8790-0973c2155718 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:09.760941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ab30c6e-3019-4303-8790-0973c2155718 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:09.793652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9ab30c6e-3019-4303-8790-0973c2155718 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048237 May 25 16:22:09.794213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 343/671] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:09 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:10.365538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b9441ffd-9607-4d0f-9989-963c09668758 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:10.366522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b9441ffd-9607-4d0f-9989-963c09668758 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:10.373878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9441ffd-9607-4d0f-9989-963c09668758 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:10.374318 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9441ffd-9607-4d0f-9989-963c09668758 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:10.375063 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9441ffd-9607-4d0f-9989-963c09668758 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:10.375524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9441ffd-9607-4d0f-9989-963c09668758 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:10.529923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a451a777-8867-4013-a72c-9de9a5624d5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:10.530867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a451a777-8867-4013-a72c-9de9a5624d5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:10.537618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a451a777-8867-4013-a72c-9de9a5624d5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:10.538218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a451a777-8867-4013-a72c-9de9a5624d5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:10.538976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a451a777-8867-4013-a72c-9de9a5624d5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:10.539439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a451a777-8867-4013-a72c-9de9a5624d5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:10.554911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:10.574972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b9441ffd-9607-4d0f-9989-963c09668758 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:10.582689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b9441ffd-9607-4d0f-9989-963c09668758 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.223621 May 25 16:22:10.583274 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 329/672] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:10 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:10.655192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2db4f8e2-d813-4673-9fdc-35a1315404a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:10.656257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2db4f8e2-d813-4673-9fdc-35a1315404a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:10.662329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2db4f8e2-d813-4673-9fdc-35a1315404a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:10.662781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2db4f8e2-d813-4673-9fdc-35a1315404a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:10.663582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2db4f8e2-d813-4673-9fdc-35a1315404a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:10.664083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2db4f8e2-d813-4673-9fdc-35a1315404a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:10.687643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a451a777-8867-4013-a72c-9de9a5624d5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:10.694265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2db4f8e2-d813-4673-9fdc-35a1315404a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042961 May 25 16:22:10.694838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 330/673] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:10 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:10.696090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a451a777-8867-4013-a72c-9de9a5624d5f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.169987 May 25 16:22:10.696650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 344/674] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:10 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:10.807034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2591c7d3-f020-4be9-afb9-ff670c7363ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:10.808207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2591c7d3-f020-4be9-afb9-ff670c7363ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:10.816036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2591c7d3-f020-4be9-afb9-ff670c7363ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:10.816606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2591c7d3-f020-4be9-afb9-ff670c7363ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:10.817543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2591c7d3-f020-4be9-afb9-ff670c7363ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:10.818087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2591c7d3-f020-4be9-afb9-ff670c7363ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:10.847484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2591c7d3-f020-4be9-afb9-ff670c7363ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045363 May 25 16:22:10.848026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 331/675] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:10 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:11.604538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5192be09-4b70-4f29-b8ec-186750bd763f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:11.605339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5192be09-4b70-4f29-b8ec-186750bd763f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:11.614240 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5192be09-4b70-4f29-b8ec-186750bd763f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:11.614807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5192be09-4b70-4f29-b8ec-186750bd763f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:11.615891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5192be09-4b70-4f29-b8ec-186750bd763f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:11.616486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5192be09-4b70-4f29-b8ec-186750bd763f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:11.707782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-97e33db6-7716-4c25-97fb-00efc7d41abc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:11.708711 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-97e33db6-7716-4c25-97fb-00efc7d41abc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:11.715656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97e33db6-7716-4c25-97fb-00efc7d41abc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:11.716143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97e33db6-7716-4c25-97fb-00efc7d41abc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:11.716913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97e33db6-7716-4c25-97fb-00efc7d41abc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:11.717348 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97e33db6-7716-4c25-97fb-00efc7d41abc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:11.719809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:11.747940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-97e33db6-7716-4c25-97fb-00efc7d41abc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045789 May 25 16:22:11.748511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 332/676] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:11 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:11.752977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a644b107-22f3-491f-8cf5-44a1e25c1e03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:11.753493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5192be09-4b70-4f29-b8ec-186750bd763f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:11.753622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a644b107-22f3-491f-8cf5-44a1e25c1e03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:11.759486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a644b107-22f3-491f-8cf5-44a1e25c1e03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:11.759933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a644b107-22f3-491f-8cf5-44a1e25c1e03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:11.760693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a644b107-22f3-491f-8cf5-44a1e25c1e03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:11.761123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a644b107-22f3-491f-8cf5-44a1e25c1e03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:11.761242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5192be09-4b70-4f29-b8ec-186750bd763f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.162600 May 25 16:22:11.761719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 345/677] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:11 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:11.859519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-56cfe8c2-6f56-4285-abc7-35549cb0ca72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:11.860550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-56cfe8c2-6f56-4285-abc7-35549cb0ca72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:11.867038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56cfe8c2-6f56-4285-abc7-35549cb0ca72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:11.867603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56cfe8c2-6f56-4285-abc7-35549cb0ca72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:11.868441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56cfe8c2-6f56-4285-abc7-35549cb0ca72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:11.868973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56cfe8c2-6f56-4285-abc7-35549cb0ca72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:11.898726 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-56cfe8c2-6f56-4285-abc7-35549cb0ca72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043370 May 25 16:22:11.899508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 346/678] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:11 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:11.907606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a644b107-22f3-491f-8cf5-44a1e25c1e03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:11.916393 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a644b107-22f3-491f-8cf5-44a1e25c1e03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.166758 May 25 16:22:11.917175 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 333/679] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:11 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:12.764046 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-76abb1b8-2fa6-4a5f-bb40-3e89de2060bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:12.765528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-76abb1b8-2fa6-4a5f-bb40-3e89de2060bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:12.775763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-76abb1b8-2fa6-4a5f-bb40-3e89de2060bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:12.776215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-76abb1b8-2fa6-4a5f-bb40-3e89de2060bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:12.776974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-76abb1b8-2fa6-4a5f-bb40-3e89de2060bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:12.777403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-76abb1b8-2fa6-4a5f-bb40-3e89de2060bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:12.778826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7e6ca1c9-9b65-4efe-bfcd-92739bf1a6b4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:12.779429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7e6ca1c9-9b65-4efe-bfcd-92739bf1a6b4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:12.792525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e6ca1c9-9b65-4efe-bfcd-92739bf1a6b4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:12.793251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e6ca1c9-9b65-4efe-bfcd-92739bf1a6b4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:12.794060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e6ca1c9-9b65-4efe-bfcd-92739bf1a6b4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:12.794501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e6ca1c9-9b65-4efe-bfcd-92739bf1a6b4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:12.809809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-76abb1b8-2fa6-4a5f-bb40-3e89de2060bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052570 May 25 16:22:12.810413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 347/680] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:12 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:12.916524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2dce1541-ea6e-4b38-973c-a1b4a2b2879b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:12.918287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2dce1541-ea6e-4b38-973c-a1b4a2b2879b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:12.929093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2dce1541-ea6e-4b38-973c-a1b4a2b2879b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:12.929997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2dce1541-ea6e-4b38-973c-a1b4a2b2879b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:12.931370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2dce1541-ea6e-4b38-973c-a1b4a2b2879b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:12.932124 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2dce1541-ea6e-4b38-973c-a1b4a2b2879b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:12.967106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2dce1541-ea6e-4b38-973c-a1b4a2b2879b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059451 May 25 16:22:12.967721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 348/681] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:12 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:12.972190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a2f8b4c4-47b0-4257-ad77-ff8d63951904 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:12.972776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a2f8b4c4-47b0-4257-ad77-ff8d63951904 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:12.978687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a2f8b4c4-47b0-4257-ad77-ff8d63951904 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:12.979123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a2f8b4c4-47b0-4257-ad77-ff8d63951904 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:12.979898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a2f8b4c4-47b0-4257-ad77-ff8d63951904 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:12.980322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a2f8b4c4-47b0-4257-ad77-ff8d63951904 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:13.161191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7e6ca1c9-9b65-4efe-bfcd-92739bf1a6b4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:13.169819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7e6ca1c9-9b65-4efe-bfcd-92739bf1a6b4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.394806 May 25 16:22:13.170393 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 334/682] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:12 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:13.180052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a2f8b4c4-47b0-4257-ad77-ff8d63951904 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:13.192235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a2f8b4c4-47b0-4257-ad77-ff8d63951904 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.223328 May 25 16:22:13.193027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 349/683] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:12 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:13.507316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cc29be90-6358-4656-9c8d-8ecb71cdacd9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:13.508131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cc29be90-6358-4656-9c8d-8ecb71cdacd9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:13.518382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc29be90-6358-4656-9c8d-8ecb71cdacd9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:13.519122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc29be90-6358-4656-9c8d-8ecb71cdacd9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:13.520436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc29be90-6358-4656-9c8d-8ecb71cdacd9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:13.521910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc29be90-6358-4656-9c8d-8ecb71cdacd9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:13.549030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cc29be90-6358-4656-9c8d-8ecb71cdacd9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Going to try to terminate instance {{(pid=117795) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:22:13.562316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:22:13.562595 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:22:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 (ip 10.222.0.45) !!! May 25 16:22:13.628753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cc29be90-6358-4656-9c8d-8ecb71cdacd9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "DELETE /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 204 len: 0 microversion: 2.1 time: 0.127692 May 25 16:22:13.629320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 335/684] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:22:13 2022] DELETE /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:22:13.640783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c80f3e40-20fa-40ee-960c-d643a6e31bea tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:13.641677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c80f3e40-20fa-40ee-960c-d643a6e31bea tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:13.650498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c80f3e40-20fa-40ee-960c-d643a6e31bea tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:13.650970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c80f3e40-20fa-40ee-960c-d643a6e31bea tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:13.651849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c80f3e40-20fa-40ee-960c-d643a6e31bea tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:13.652324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c80f3e40-20fa-40ee-960c-d643a6e31bea tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:13.789984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c80f3e40-20fa-40ee-960c-d643a6e31bea tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:13.797907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c80f3e40-20fa-40ee-960c-d643a6e31bea tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.163625 May 25 16:22:13.798504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 350/685] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:13 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:13.823487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c5947062-aac4-4ddd-84e9-20495082cb2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:13.824438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c5947062-aac4-4ddd-84e9-20495082cb2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:13.831823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5947062-aac4-4ddd-84e9-20495082cb2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:13.832264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5947062-aac4-4ddd-84e9-20495082cb2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:13.833028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5947062-aac4-4ddd-84e9-20495082cb2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:13.833463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5947062-aac4-4ddd-84e9-20495082cb2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:13.862103 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c5947062-aac4-4ddd-84e9-20495082cb2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043191 May 25 16:22:13.862641 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 336/686] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:13 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:22:13.981280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4da46da1-688d-4052-95d6-4e3dababcb8a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:13.982355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4da46da1-688d-4052-95d6-4e3dababcb8a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:13.989556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4da46da1-688d-4052-95d6-4e3dababcb8a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:13.990054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4da46da1-688d-4052-95d6-4e3dababcb8a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:13.990890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4da46da1-688d-4052-95d6-4e3dababcb8a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:13.991411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4da46da1-688d-4052-95d6-4e3dababcb8a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:14.022692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4da46da1-688d-4052-95d6-4e3dababcb8a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046062 May 25 16:22:14.023361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 351/687] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:13 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:14.192779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d72b11e7-6e77-4700-bd13-0e05c46eff1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:14.193667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d72b11e7-6e77-4700-bd13-0e05c46eff1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:14.205832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d72b11e7-6e77-4700-bd13-0e05c46eff1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:14.206624 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d72b11e7-6e77-4700-bd13-0e05c46eff1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:14.207933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d72b11e7-6e77-4700-bd13-0e05c46eff1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:14.208850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d72b11e7-6e77-4700-bd13-0e05c46eff1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:14.212727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a459a795-4773-46ce-b76e-3d093bb15580 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:14.213342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a459a795-4773-46ce-b76e-3d093bb15580 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:14.219741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a459a795-4773-46ce-b76e-3d093bb15580 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:14.220196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a459a795-4773-46ce-b76e-3d093bb15580 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:14.220944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a459a795-4773-46ce-b76e-3d093bb15580 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:14.221406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a459a795-4773-46ce-b76e-3d093bb15580 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:14.411115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d72b11e7-6e77-4700-bd13-0e05c46eff1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:14.419891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d72b11e7-6e77-4700-bd13-0e05c46eff1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.233722 May 25 16:22:14.420585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 337/688] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:14 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:14.427132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a459a795-4773-46ce-b76e-3d093bb15580 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:14.435435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a459a795-4773-46ce-b76e-3d093bb15580 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.228292 May 25 16:22:14.436003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 352/689] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:14 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:14.819432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-45796893-e07f-4877-914a-78525db2b1c0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:14.820615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-45796893-e07f-4877-914a-78525db2b1c0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:14.831451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45796893-e07f-4877-914a-78525db2b1c0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:14.832408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45796893-e07f-4877-914a-78525db2b1c0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:14.833404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45796893-e07f-4877-914a-78525db2b1c0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:14.834116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45796893-e07f-4877-914a-78525db2b1c0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:14.873768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ea00f367-4f55-4176-8207-12699c279a54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:14.874691 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ea00f367-4f55-4176-8207-12699c279a54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:14.881639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea00f367-4f55-4176-8207-12699c279a54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:14.882066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea00f367-4f55-4176-8207-12699c279a54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:14.882839 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea00f367-4f55-4176-8207-12699c279a54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:14.883313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea00f367-4f55-4176-8207-12699c279a54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:14.913944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ea00f367-4f55-4176-8207-12699c279a54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044044 May 25 16:22:14.914519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 353/690] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:14 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:14.973079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-45796893-e07f-4877-914a-78525db2b1c0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:14.985498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-45796893-e07f-4877-914a-78525db2b1c0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.172131 May 25 16:22:14.986280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 338/691] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:14 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:15.034173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4bb7c391-14dc-4579-a345-61df0be8c5e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:15.035102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4bb7c391-14dc-4579-a345-61df0be8c5e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:15.042296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bb7c391-14dc-4579-a345-61df0be8c5e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:15.042750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bb7c391-14dc-4579-a345-61df0be8c5e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:15.043556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bb7c391-14dc-4579-a345-61df0be8c5e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:15.043994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bb7c391-14dc-4579-a345-61df0be8c5e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:15.075960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4bb7c391-14dc-4579-a345-61df0be8c5e2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045767 May 25 16:22:15.076785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 354/692] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:15 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:15.439821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3eead254-7c6d-48a3-a3f1-74cbfdcafb80 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:15.440694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3eead254-7c6d-48a3-a3f1-74cbfdcafb80 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:15.448518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3eead254-7c6d-48a3-a3f1-74cbfdcafb80 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:15.449028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3eead254-7c6d-48a3-a3f1-74cbfdcafb80 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:15.449802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3eead254-7c6d-48a3-a3f1-74cbfdcafb80 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:15.450242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3eead254-7c6d-48a3-a3f1-74cbfdcafb80 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:15.452837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5e6a5ab3-61da-43f5-8765-64efcd38f1e7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:15.453440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5e6a5ab3-61da-43f5-8765-64efcd38f1e7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:15.459837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e6a5ab3-61da-43f5-8765-64efcd38f1e7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:15.460282 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e6a5ab3-61da-43f5-8765-64efcd38f1e7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:15.461169 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e6a5ab3-61da-43f5-8765-64efcd38f1e7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:15.461593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e6a5ab3-61da-43f5-8765-64efcd38f1e7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:15.597538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3eead254-7c6d-48a3-a3f1-74cbfdcafb80 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:15.605239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3eead254-7c6d-48a3-a3f1-74cbfdcafb80 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.171047 May 25 16:22:15.605753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 339/693] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:15 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:15.698339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5e6a5ab3-61da-43f5-8765-64efcd38f1e7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:15.709231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5e6a5ab3-61da-43f5-8765-64efcd38f1e7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.259897 May 25 16:22:15.710093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 355/694] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:15 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:15.926599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e4bb15fd-0c11-4bbb-868d-16f681a5a65b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:15.927691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e4bb15fd-0c11-4bbb-868d-16f681a5a65b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:15.934927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4bb15fd-0c11-4bbb-868d-16f681a5a65b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:15.935431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4bb15fd-0c11-4bbb-868d-16f681a5a65b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:15.936218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4bb15fd-0c11-4bbb-868d-16f681a5a65b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:15.936677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4bb15fd-0c11-4bbb-868d-16f681a5a65b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:15.968141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e4bb15fd-0c11-4bbb-868d-16f681a5a65b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045673 May 25 16:22:15.968607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 340/695] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:15 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:16.005526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ad38eeb2-985a-4afe-9447-31439229fe49 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:16.006195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ad38eeb2-985a-4afe-9447-31439229fe49 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:16.013045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad38eeb2-985a-4afe-9447-31439229fe49 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.013501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad38eeb2-985a-4afe-9447-31439229fe49 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:16.014264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad38eeb2-985a-4afe-9447-31439229fe49 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.014697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad38eeb2-985a-4afe-9447-31439229fe49 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:16.088260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-370d875d-6409-4911-98d3-1eac47302daf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:16.089186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-370d875d-6409-4911-98d3-1eac47302daf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:16.095613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-370d875d-6409-4911-98d3-1eac47302daf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.096064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-370d875d-6409-4911-98d3-1eac47302daf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:16.096812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-370d875d-6409-4911-98d3-1eac47302daf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.097241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-370d875d-6409-4911-98d3-1eac47302daf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:16.131861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-370d875d-6409-4911-98d3-1eac47302daf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047595 May 25 16:22:16.132629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 341/696] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:16 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:16.156707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ad38eeb2-985a-4afe-9447-31439229fe49 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:16.164617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ad38eeb2-985a-4afe-9447-31439229fe49 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.163394 May 25 16:22:16.165176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 356/697] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:16 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:16.623442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-961dab30-f9c6-4021-ae75-66a5865c87b5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:16.624048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-961dab30-f9c6-4021-ae75-66a5865c87b5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:16.630414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-961dab30-f9c6-4021-ae75-66a5865c87b5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.630869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-961dab30-f9c6-4021-ae75-66a5865c87b5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:16.631659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-961dab30-f9c6-4021-ae75-66a5865c87b5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.632093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-961dab30-f9c6-4021-ae75-66a5865c87b5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:16.727489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6cff1a53-53e6-48b2-97c6-9db675aa8acb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:16.728060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6cff1a53-53e6-48b2-97c6-9db675aa8acb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:16.734160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cff1a53-53e6-48b2-97c6-9db675aa8acb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.734605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cff1a53-53e6-48b2-97c6-9db675aa8acb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:16.735389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cff1a53-53e6-48b2-97c6-9db675aa8acb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.735817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cff1a53-53e6-48b2-97c6-9db675aa8acb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:16.782500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-961dab30-f9c6-4021-ae75-66a5865c87b5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:16.790724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-961dab30-f9c6-4021-ae75-66a5865c87b5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.170878 May 25 16:22:16.791286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 342/698] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:16 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:16.891628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6cff1a53-53e6-48b2-97c6-9db675aa8acb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:16.904177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6cff1a53-53e6-48b2-97c6-9db675aa8acb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.180851 May 25 16:22:16.904953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 357/699] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:16 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:16.982629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-98da58f0-6809-443c-9c09-c7013ec2893b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:16.983883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-98da58f0-6809-443c-9c09-c7013ec2893b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:16.993008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98da58f0-6809-443c-9c09-c7013ec2893b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.993605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98da58f0-6809-443c-9c09-c7013ec2893b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:16.994609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98da58f0-6809-443c-9c09-c7013ec2893b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:16.995204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98da58f0-6809-443c-9c09-c7013ec2893b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:17.028037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-98da58f0-6809-443c-9c09-c7013ec2893b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050711 May 25 16:22:17.028605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 343/700] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:16 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:17.148343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-517006d7-a88a-4c87-b01c-f458c0a42a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:17.149273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-517006d7-a88a-4c87-b01c-f458c0a42a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:17.155929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-517006d7-a88a-4c87-b01c-f458c0a42a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:17.156429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-517006d7-a88a-4c87-b01c-f458c0a42a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:17.157125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-517006d7-a88a-4c87-b01c-f458c0a42a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:17.157562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-517006d7-a88a-4c87-b01c-f458c0a42a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:17.181853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b834afac-7723-4db4-8a74-070d3d26b35f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:17.182461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b834afac-7723-4db4-8a74-070d3d26b35f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:17.188629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b834afac-7723-4db4-8a74-070d3d26b35f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:17.188788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-517006d7-a88a-4c87-b01c-f458c0a42a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047305 May 25 16:22:17.189078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b834afac-7723-4db4-8a74-070d3d26b35f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:17.189365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 358/701] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:17 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:17.189825 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b834afac-7723-4db4-8a74-070d3d26b35f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:17.190259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b834afac-7723-4db4-8a74-070d3d26b35f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:17.334159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b834afac-7723-4db4-8a74-070d3d26b35f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:17.342466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b834afac-7723-4db4-8a74-070d3d26b35f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.164536 May 25 16:22:17.343138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 344/702] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:17 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:17.809794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fd805070-0596-4e98-ad22-a0da1b27a438 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:17.810597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fd805070-0596-4e98-ad22-a0da1b27a438 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:17.819691 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd805070-0596-4e98-ad22-a0da1b27a438 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:17.820292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd805070-0596-4e98-ad22-a0da1b27a438 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:17.821383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd805070-0596-4e98-ad22-a0da1b27a438 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:17.822130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd805070-0596-4e98-ad22-a0da1b27a438 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:17.930816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-872e5e92-ff28-47ab-aad2-f0e0d88aad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:17.932327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-872e5e92-ff28-47ab-aad2-f0e0d88aad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:17.938738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-872e5e92-ff28-47ab-aad2-f0e0d88aad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:17.939184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-872e5e92-ff28-47ab-aad2-f0e0d88aad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:17.939974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-872e5e92-ff28-47ab-aad2-f0e0d88aad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:17.940431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-872e5e92-ff28-47ab-aad2-f0e0d88aad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:17.963744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fd805070-0596-4e98-ad22-a0da1b27a438 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:17.971528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fd805070-0596-4e98-ad22-a0da1b27a438 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.167274 May 25 16:22:17.972298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 359/703] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:17 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:18.040014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-89b12200-56d5-4d7e-9dde-b6ea1f186a24 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:18.041312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-89b12200-56d5-4d7e-9dde-b6ea1f186a24 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:18.047346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89b12200-56d5-4d7e-9dde-b6ea1f186a24 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:18.047799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89b12200-56d5-4d7e-9dde-b6ea1f186a24 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:18.048549 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89b12200-56d5-4d7e-9dde-b6ea1f186a24 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:18.048976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89b12200-56d5-4d7e-9dde-b6ea1f186a24 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:18.078950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-89b12200-56d5-4d7e-9dde-b6ea1f186a24 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042968 May 25 16:22:18.079775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 360/704] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:18 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:22:18.084023 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-872e5e92-ff28-47ab-aad2-f0e0d88aad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:18.092503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-872e5e92-ff28-47ab-aad2-f0e0d88aad0b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.170935 May 25 16:22:18.093064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 345/705] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:17 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:18.201331 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a23112fc-fa36-4a9b-bb55-242e5a62e7f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:18.202244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a23112fc-fa36-4a9b-bb55-242e5a62e7f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:18.209089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a23112fc-fa36-4a9b-bb55-242e5a62e7f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:18.209528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a23112fc-fa36-4a9b-bb55-242e5a62e7f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:18.210277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a23112fc-fa36-4a9b-bb55-242e5a62e7f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:18.210918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a23112fc-fa36-4a9b-bb55-242e5a62e7f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:18.213276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:18.242992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a23112fc-fa36-4a9b-bb55-242e5a62e7f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045741 May 25 16:22:18.243562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 361/706] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:18.366436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5151e243-77dc-4285-95da-2f10cb5ebf86 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:18.367119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5151e243-77dc-4285-95da-2f10cb5ebf86 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:18.373987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5151e243-77dc-4285-95da-2f10cb5ebf86 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:18.374472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5151e243-77dc-4285-95da-2f10cb5ebf86 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:18.375370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5151e243-77dc-4285-95da-2f10cb5ebf86 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:18.375809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5151e243-77dc-4285-95da-2f10cb5ebf86 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:18.516426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5151e243-77dc-4285-95da-2f10cb5ebf86 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:18.524621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5151e243-77dc-4285-95da-2f10cb5ebf86 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.162196 May 25 16:22:18.525217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 346/707] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:18 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:18.992207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-df2f1f64-8fea-43c1-a3e0-55488b7d351a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:18.993052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-df2f1f64-8fea-43c1-a3e0-55488b7d351a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:19.003121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df2f1f64-8fea-43c1-a3e0-55488b7d351a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.003802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df2f1f64-8fea-43c1-a3e0-55488b7d351a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.004939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df2f1f64-8fea-43c1-a3e0-55488b7d351a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.005550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df2f1f64-8fea-43c1-a3e0-55488b7d351a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.090387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4450ffbe-5442-4e17-b63d-899218830829 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:19.091433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4450ffbe-5442-4e17-b63d-899218830829 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:19.098274 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4450ffbe-5442-4e17-b63d-899218830829 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.098727 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4450ffbe-5442-4e17-b63d-899218830829 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.099533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4450ffbe-5442-4e17-b63d-899218830829 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.099970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4450ffbe-5442-4e17-b63d-899218830829 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.102677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:19.133364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4450ffbe-5442-4e17-b63d-899218830829 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046914 May 25 16:22:19.134182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 347/708] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:19 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:19.138420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6ad3bfa6-ce01-468b-a174-358930ec8985 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:19.139020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6ad3bfa6-ce01-468b-a174-358930ec8985 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:19.145346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ad3bfa6-ce01-468b-a174-358930ec8985 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.145799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ad3bfa6-ce01-468b-a174-358930ec8985 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.146566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ad3bfa6-ce01-468b-a174-358930ec8985 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.146982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ad3bfa6-ce01-468b-a174-358930ec8985 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.154433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-df2f1f64-8fea-43c1-a3e0-55488b7d351a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:19.162108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-df2f1f64-8fea-43c1-a3e0-55488b7d351a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.175921 May 25 16:22:19.162696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 362/709] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:18 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:19.254553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-098c9509-29c8-43cc-bab2-7a34e26c158c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:19.255460 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-098c9509-29c8-43cc-bab2-7a34e26c158c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:19.262607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-098c9509-29c8-43cc-bab2-7a34e26c158c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.263230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-098c9509-29c8-43cc-bab2-7a34e26c158c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.263954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-098c9509-29c8-43cc-bab2-7a34e26c158c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.264387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-098c9509-29c8-43cc-bab2-7a34e26c158c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.292319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6ad3bfa6-ce01-468b-a174-358930ec8985 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:19.297330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-098c9509-29c8-43cc-bab2-7a34e26c158c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046936 May 25 16:22:19.297963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 363/710] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:19 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:22:19.300702 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6ad3bfa6-ce01-468b-a174-358930ec8985 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.165734 May 25 16:22:19.301240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 348/711] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:19 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:19.548740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3c05ad97-6692-4f54-aa1d-acf83c6774ae tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:19.549736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3c05ad97-6692-4f54-aa1d-acf83c6774ae tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:19.557946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c05ad97-6692-4f54-aa1d-acf83c6774ae tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.558398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c05ad97-6692-4f54-aa1d-acf83c6774ae tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.559175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c05ad97-6692-4f54-aa1d-acf83c6774ae tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:19.559622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c05ad97-6692-4f54-aa1d-acf83c6774ae tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:19.703465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3c05ad97-6692-4f54-aa1d-acf83c6774ae tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:19.711280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3c05ad97-6692-4f54-aa1d-acf83c6774ae tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.169070 May 25 16:22:19.712035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 364/712] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:19 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:20.149108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b4ddc10a-cafa-4023-a558-3187992f53b5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:20.150040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b4ddc10a-cafa-4023-a558-3187992f53b5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:20.152498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:20.157850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b4ddc10a-cafa-4023-a558-3187992f53b5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.158287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b4ddc10a-cafa-4023-a558-3187992f53b5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.159043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b4ddc10a-cafa-4023-a558-3187992f53b5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.159494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b4ddc10a-cafa-4023-a558-3187992f53b5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.179757 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-70dfcb57-707c-47d1-bc47-27e21d0afe19 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:20.180557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-70dfcb57-707c-47d1-bc47-27e21d0afe19 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:20.186557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70dfcb57-707c-47d1-bc47-27e21d0afe19 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.187005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70dfcb57-707c-47d1-bc47-27e21d0afe19 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.187801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70dfcb57-707c-47d1-bc47-27e21d0afe19 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.188234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70dfcb57-707c-47d1-bc47-27e21d0afe19 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.189998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b4ddc10a-cafa-4023-a558-3187992f53b5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047230 May 25 16:22:20.190869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 349/713] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:20 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:20.309831 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-22df8b29-76e9-49f0-b074-27a6882d935f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:20.310916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-22df8b29-76e9-49f0-b074-27a6882d935f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:20.317156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22df8b29-76e9-49f0-b074-27a6882d935f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.317614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22df8b29-76e9-49f0-b074-27a6882d935f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.318438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22df8b29-76e9-49f0-b074-27a6882d935f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.318813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22df8b29-76e9-49f0-b074-27a6882d935f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.321151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-70dfcb57-707c-47d1-bc47-27e21d0afe19 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:20.328917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-70dfcb57-707c-47d1-bc47-27e21d0afe19 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.153097 May 25 16:22:20.329526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 365/714] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:20 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:20.335086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a18f82b6-1382-4de5-8949-d82345f7d0df tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:20.335974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a18f82b6-1382-4de5-8949-d82345f7d0df tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:20.342817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a18f82b6-1382-4de5-8949-d82345f7d0df tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.343263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a18f82b6-1382-4de5-8949-d82345f7d0df tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.344054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a18f82b6-1382-4de5-8949-d82345f7d0df tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.344483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a18f82b6-1382-4de5-8949-d82345f7d0df tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.350114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-22df8b29-76e9-49f0-b074-27a6882d935f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044425 May 25 16:22:20.350673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 350/715] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:20 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:22:20.495602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a18f82b6-1382-4de5-8949-d82345f7d0df tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:20.503632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a18f82b6-1382-4de5-8949-d82345f7d0df tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.173071 May 25 16:22:20.504225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 366/716] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:20 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:20.729858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-229a1a96-f355-442d-85bd-e6bb1672b6e7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:20.730595 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-229a1a96-f355-442d-85bd-e6bb1672b6e7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:20.739459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-229a1a96-f355-442d-85bd-e6bb1672b6e7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.740105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-229a1a96-f355-442d-85bd-e6bb1672b6e7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.741186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-229a1a96-f355-442d-85bd-e6bb1672b6e7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:20.741877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-229a1a96-f355-442d-85bd-e6bb1672b6e7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:20.878273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-229a1a96-f355-442d-85bd-e6bb1672b6e7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:20.890827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-229a1a96-f355-442d-85bd-e6bb1672b6e7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.165738 May 25 16:22:20.891935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 351/717] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:20 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:21.205945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d7c88161-4ca4-4e74-8091-3281a5a4cf65 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:21.207569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d7c88161-4ca4-4e74-8091-3281a5a4cf65 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:21.217916 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d7c88161-4ca4-4e74-8091-3281a5a4cf65 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.218659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d7c88161-4ca4-4e74-8091-3281a5a4cf65 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:21.219958 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d7c88161-4ca4-4e74-8091-3281a5a4cf65 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.220687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d7c88161-4ca4-4e74-8091-3281a5a4cf65 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:21.252525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d7c88161-4ca4-4e74-8091-3281a5a4cf65 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053265 May 25 16:22:21.253071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 367/718] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:21 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:21.349066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-36036111-743c-4458-bfd5-b4c34b8f9704 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:21.349805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-36036111-743c-4458-bfd5-b4c34b8f9704 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:21.357220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-36036111-743c-4458-bfd5-b4c34b8f9704 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.357670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-36036111-743c-4458-bfd5-b4c34b8f9704 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:21.358433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-36036111-743c-4458-bfd5-b4c34b8f9704 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.358863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-36036111-743c-4458-bfd5-b4c34b8f9704 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:21.361580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b30c7cd8-4d07-422d-84b5-ff3a57841c09 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:21.362492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b30c7cd8-4d07-422d-84b5-ff3a57841c09 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:21.369444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b30c7cd8-4d07-422d-84b5-ff3a57841c09 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.369885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b30c7cd8-4d07-422d-84b5-ff3a57841c09 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:21.370638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b30c7cd8-4d07-422d-84b5-ff3a57841c09 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.371056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b30c7cd8-4d07-422d-84b5-ff3a57841c09 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:21.400919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b30c7cd8-4d07-422d-84b5-ff3a57841c09 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043226 May 25 16:22:21.401446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 368/719] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:21 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:21.493796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-36036111-743c-4458-bfd5-b4c34b8f9704 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:21.505526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-36036111-743c-4458-bfd5-b4c34b8f9704 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.161697 May 25 16:22:21.506325 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 352/720] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:21 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:21.521034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bc803975-1eda-4933-82d0-8206c6cf5172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:21.521632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bc803975-1eda-4933-82d0-8206c6cf5172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:21.523550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:21.529569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc803975-1eda-4933-82d0-8206c6cf5172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.530000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc803975-1eda-4933-82d0-8206c6cf5172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:21.530740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc803975-1eda-4933-82d0-8206c6cf5172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.531180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc803975-1eda-4933-82d0-8206c6cf5172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:21.674847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bc803975-1eda-4933-82d0-8206c6cf5172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:21.684953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bc803975-1eda-4933-82d0-8206c6cf5172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.167764 May 25 16:22:21.685529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 369/721] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:21 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:21.911261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-233039b2-b54e-445e-9b95-e084462954c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:21.911877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-233039b2-b54e-445e-9b95-e084462954c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:21.919475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-233039b2-b54e-445e-9b95-e084462954c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.919944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-233039b2-b54e-445e-9b95-e084462954c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:21.920710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-233039b2-b54e-445e-9b95-e084462954c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:21.921147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-233039b2-b54e-445e-9b95-e084462954c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:22.053750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-233039b2-b54e-445e-9b95-e084462954c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:22.061978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-233039b2-b54e-445e-9b95-e084462954c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.154804 May 25 16:22:22.062560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 353/722] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:21 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:22.271237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4eb77aa8-cfb8-443c-b5d1-6edad8c55717 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:22.272602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4eb77aa8-cfb8-443c-b5d1-6edad8c55717 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:22.279263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4eb77aa8-cfb8-443c-b5d1-6edad8c55717 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:22.279712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4eb77aa8-cfb8-443c-b5d1-6edad8c55717 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:22.280459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4eb77aa8-cfb8-443c-b5d1-6edad8c55717 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:22.280877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4eb77aa8-cfb8-443c-b5d1-6edad8c55717 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:22.311454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4eb77aa8-cfb8-443c-b5d1-6edad8c55717 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049629 May 25 16:22:22.312008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 370/723] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:22 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:22.417452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7e18fefc-2f78-4fe9-a34c-dc8fb4ba6e71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:22.419057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7e18fefc-2f78-4fe9-a34c-dc8fb4ba6e71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:22.429830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e18fefc-2f78-4fe9-a34c-dc8fb4ba6e71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:22.430266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e18fefc-2f78-4fe9-a34c-dc8fb4ba6e71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:22.431027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e18fefc-2f78-4fe9-a34c-dc8fb4ba6e71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:22.431485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7e18fefc-2f78-4fe9-a34c-dc8fb4ba6e71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:22.461172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7e18fefc-2f78-4fe9-a34c-dc8fb4ba6e71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051025 May 25 16:22:22.461711 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 354/724] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:22 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:22.523631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b0f77367-61d9-4dd5-b350-ed9cb11d4a96 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:22.524234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b0f77367-61d9-4dd5-b350-ed9cb11d4a96 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:22.531174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0f77367-61d9-4dd5-b350-ed9cb11d4a96 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:22.531625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0f77367-61d9-4dd5-b350-ed9cb11d4a96 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:22.532372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0f77367-61d9-4dd5-b350-ed9cb11d4a96 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:22.532803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0f77367-61d9-4dd5-b350-ed9cb11d4a96 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:22.665167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b0f77367-61d9-4dd5-b350-ed9cb11d4a96 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:22.673114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b0f77367-61d9-4dd5-b350-ed9cb11d4a96 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.153503 May 25 16:22:22.673676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 371/725] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:22 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:22.702166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-56529bb3-078a-4608-9d63-eda107ac6889 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:22.702797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-56529bb3-078a-4608-9d63-eda107ac6889 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:22.708918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-56529bb3-078a-4608-9d63-eda107ac6889 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:22.709368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-56529bb3-078a-4608-9d63-eda107ac6889 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:22.710126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-56529bb3-078a-4608-9d63-eda107ac6889 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:22.710598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-56529bb3-078a-4608-9d63-eda107ac6889 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:22.861400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-56529bb3-078a-4608-9d63-eda107ac6889 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:22.871647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-56529bb3-078a-4608-9d63-eda107ac6889 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1756 microversion: 2.1 time: 0.173595 May 25 16:22:22.872474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 355/726] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:22 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1756 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:23.083723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f2f16388-27b9-4030-8f58-147fe7ef3f11 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:23.084745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f2f16388-27b9-4030-8f58-147fe7ef3f11 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:23.091807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2f16388-27b9-4030-8f58-147fe7ef3f11 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.092259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2f16388-27b9-4030-8f58-147fe7ef3f11 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:23.093025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2f16388-27b9-4030-8f58-147fe7ef3f11 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.093457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2f16388-27b9-4030-8f58-147fe7ef3f11 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:23.230077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f2f16388-27b9-4030-8f58-147fe7ef3f11 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:23.238381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f2f16388-27b9-4030-8f58-147fe7ef3f11 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.161305 May 25 16:22:23.238958 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 372/727] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:23 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:23.324030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2969d591-a394-4b11-8e91-8b62193b5c35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:23.325011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2969d591-a394-4b11-8e91-8b62193b5c35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:23.332114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2969d591-a394-4b11-8e91-8b62193b5c35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.332632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2969d591-a394-4b11-8e91-8b62193b5c35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:23.333465 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2969d591-a394-4b11-8e91-8b62193b5c35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.333945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2969d591-a394-4b11-8e91-8b62193b5c35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:23.364192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2969d591-a394-4b11-8e91-8b62193b5c35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044515 May 25 16:22:23.364766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 356/728] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:23 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:23.605230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-eb6b7b42-ef9d-4d68-9d4e-0aa0c4fe3bc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:23.606360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-eb6b7b42-ef9d-4d68-9d4e-0aa0c4fe3bc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:23.614027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb6b7b42-ef9d-4d68-9d4e-0aa0c4fe3bc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.614644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb6b7b42-ef9d-4d68-9d4e-0aa0c4fe3bc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:23.615661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb6b7b42-ef9d-4d68-9d4e-0aa0c4fe3bc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.616166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb6b7b42-ef9d-4d68-9d4e-0aa0c4fe3bc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:23.649204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-eb6b7b42-ef9d-4d68-9d4e-0aa0c4fe3bc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.179789 May 25 16:22:23.649759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 373/729] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:23 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:23.843535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-317a75a2-ea36-4ce7-bf2a-cf70093e0942 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:23.844345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-317a75a2-ea36-4ce7-bf2a-cf70093e0942 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:23.852639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-317a75a2-ea36-4ce7-bf2a-cf70093e0942 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.853138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-317a75a2-ea36-4ce7-bf2a-cf70093e0942 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:23.853983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-317a75a2-ea36-4ce7-bf2a-cf70093e0942 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.854423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-317a75a2-ea36-4ce7-bf2a-cf70093e0942 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:23.900668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-afbe900a-6cc8-4223-9360-3a7ad7a31987 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:23.901291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-afbe900a-6cc8-4223-9360-3a7ad7a31987 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:23.908459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-afbe900a-6cc8-4223-9360-3a7ad7a31987 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.908903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-afbe900a-6cc8-4223-9360-3a7ad7a31987 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:23.909854 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-afbe900a-6cc8-4223-9360-3a7ad7a31987 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:23.910255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-afbe900a-6cc8-4223-9360-3a7ad7a31987 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:24.154378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-317a75a2-ea36-4ce7-bf2a-cf70093e0942 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:24.162153 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-317a75a2-ea36-4ce7-bf2a-cf70093e0942 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.475747 May 25 16:22:24.162738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 357/730] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:23 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:24.218590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-afbe900a-6cc8-4223-9360-3a7ad7a31987 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:24.230040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-afbe900a-6cc8-4223-9360-3a7ad7a31987 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1759 microversion: 2.1 time: 0.344264 May 25 16:22:24.230618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 374/731] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:23 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1759 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:24.255918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-425f758f-b786-4e8d-8188-24fd34e84667 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:24.256562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-425f758f-b786-4e8d-8188-24fd34e84667 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:24.263015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-425f758f-b786-4e8d-8188-24fd34e84667 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:24.263504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-425f758f-b786-4e8d-8188-24fd34e84667 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:24.264279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-425f758f-b786-4e8d-8188-24fd34e84667 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:24.264720 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-425f758f-b786-4e8d-8188-24fd34e84667 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:24.376276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0677681e-90d8-41db-bfce-1be1e45c76c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:24.377603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0677681e-90d8-41db-bfce-1be1e45c76c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:24.384370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0677681e-90d8-41db-bfce-1be1e45c76c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:24.384801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0677681e-90d8-41db-bfce-1be1e45c76c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:24.385591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0677681e-90d8-41db-bfce-1be1e45c76c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:24.386029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0677681e-90d8-41db-bfce-1be1e45c76c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:24.404544 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-425f758f-b786-4e8d-8188-24fd34e84667 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:24.412582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-425f758f-b786-4e8d-8188-24fd34e84667 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.160685 May 25 16:22:24.413156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 358/732] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:24 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:24.416868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0677681e-90d8-41db-bfce-1be1e45c76c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044594 May 25 16:22:24.417408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 375/733] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:24 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:24.662277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c7673d85-cdae-4554-bb45-a8512d43ce7e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:24.663383 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c7673d85-cdae-4554-bb45-a8512d43ce7e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:24.670286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7673d85-cdae-4554-bb45-a8512d43ce7e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:24.670761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7673d85-cdae-4554-bb45-a8512d43ce7e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:24.671665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7673d85-cdae-4554-bb45-a8512d43ce7e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:24.672221 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7673d85-cdae-4554-bb45-a8512d43ce7e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:24.704482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c7673d85-cdae-4554-bb45-a8512d43ce7e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047075 May 25 16:22:24.705045 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 359/734] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:24 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:25.185313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0ddb1024-c694-4f51-8249-55f5183c2b85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:25.185911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0ddb1024-c694-4f51-8249-55f5183c2b85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:25.196187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ddb1024-c694-4f51-8249-55f5183c2b85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:25.196633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ddb1024-c694-4f51-8249-55f5183c2b85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:25.197404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ddb1024-c694-4f51-8249-55f5183c2b85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:25.197834 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ddb1024-c694-4f51-8249-55f5183c2b85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:25.429678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a9e9ac80-3e3e-4e4b-8511-ba89e9a6acc9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:25.430691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a9e9ac80-3e3e-4e4b-8511-ba89e9a6acc9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:25.437799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9e9ac80-3e3e-4e4b-8511-ba89e9a6acc9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:25.438240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9e9ac80-3e3e-4e4b-8511-ba89e9a6acc9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:25.439867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9e9ac80-3e3e-4e4b-8511-ba89e9a6acc9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:25.440314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9e9ac80-3e3e-4e4b-8511-ba89e9a6acc9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:25.473824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a9e9ac80-3e3e-4e4b-8511-ba89e9a6acc9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048942 May 25 16:22:25.475742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 360/735] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:25 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:25.482919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-186faded-5645-4ec4-bf3e-abde6e66f66d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:25.483817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-186faded-5645-4ec4-bf3e-abde6e66f66d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:25.489839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-186faded-5645-4ec4-bf3e-abde6e66f66d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:25.490285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-186faded-5645-4ec4-bf3e-abde6e66f66d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:25.491053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-186faded-5645-4ec4-bf3e-abde6e66f66d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:25.491655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-186faded-5645-4ec4-bf3e-abde6e66f66d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:25.563401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ddb1024-c694-4f51-8249-55f5183c2b85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:25.572835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0ddb1024-c694-4f51-8249-55f5183c2b85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1674 microversion: 2.1 time: 0.394690 May 25 16:22:25.582643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 376/736] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:25 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1674 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:25.718989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b110b422-8031-42c4-ab91-3177609e6a1e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:25.720232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b110b422-8031-42c4-ab91-3177609e6a1e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:25.728651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b110b422-8031-42c4-ab91-3177609e6a1e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:25.729163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b110b422-8031-42c4-ab91-3177609e6a1e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:25.730005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b110b422-8031-42c4-ab91-3177609e6a1e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:25.730511 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b110b422-8031-42c4-ab91-3177609e6a1e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:25.764462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:25.770437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b110b422-8031-42c4-ab91-3177609e6a1e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057382 May 25 16:22:25.775590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 377/737] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:25 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:25.807945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-186faded-5645-4ec4-bf3e-abde6e66f66d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:25.816904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-186faded-5645-4ec4-bf3e-abde6e66f66d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1577 microversion: 2.1 time: 0.341434 May 25 16:22:25.817696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 361/738] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:25 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1577 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:26.152651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c3fe8061-5274-4734-a9c8-454e65eed4d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:26.154279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c3fe8061-5274-4734-a9c8-454e65eed4d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:26.164983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3fe8061-5274-4734-a9c8-454e65eed4d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.165735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3fe8061-5274-4734-a9c8-454e65eed4d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.166493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3fe8061-5274-4734-a9c8-454e65eed4d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.166914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3fe8061-5274-4734-a9c8-454e65eed4d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.193405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-c3fe8061-5274-4734-a9c8-454e65eed4d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lazy-loading 'flavor' on Instance uuid 621e54b1-cdde-493f-9bad-1c1222a0731a {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:22:26.241838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-3cd14f00-5116-4f38-91f5-1101f43d97fc req-d6fb8cf0-41e3-465b-917a-b036ebb87a52 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b3b920a7-fa28-4150-b889-d9a2ea63bc16", "name": "network-vif-deleted", "tag": "533fafe5-1581-4d83-887b-681b728ba38c"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:26.281455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-3cd14f00-5116-4f38-91f5-1101f43d97fc req-d6fb8cf0-41e3-465b-917a-b036ebb87a52 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.281677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-3cd14f00-5116-4f38-91f5-1101f43d97fc req-d6fb8cf0-41e3-465b-917a-b036ebb87a52 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.304592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-3cd14f00-5116-4f38-91f5-1101f43d97fc req-d6fb8cf0-41e3-465b-917a-b036ebb87a52 service nova] Creating event network-vif-deleted:533fafe5-1581-4d83-887b-681b728ba38c for instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 on hv-cinder-84327 May 25 16:22:26.310642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-3cd14f00-5116-4f38-91f5-1101f43d97fc req-d6fb8cf0-41e3-465b-917a-b036ebb87a52 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075788 May 25 16:22:26.311520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 362/739] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:22:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:26.416552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c3fe8061-5274-4734-a9c8-454e65eed4d8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/action" status: 200 len: 32 microversion: 2.1 time: 0.268552 May 25 16:22:26.421163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 378/740] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:22:26 2022] POST /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/action => generated 32 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 25 16:22:26.429512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9713f023-0278-439f-8187-6f139374fd60 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:26.430336 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9713f023-0278-439f-8187-6f139374fd60 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:26.439196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9713f023-0278-439f-8187-6f139374fd60 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.439962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9713f023-0278-439f-8187-6f139374fd60 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.440982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9713f023-0278-439f-8187-6f139374fd60 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.441561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9713f023-0278-439f-8187-6f139374fd60 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.486150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1c809f35-d457-45be-893e-24251b185686 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:26.487409 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1c809f35-d457-45be-893e-24251b185686 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:26.500696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1c809f35-d457-45be-893e-24251b185686 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.501242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1c809f35-d457-45be-893e-24251b185686 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.502290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1c809f35-d457-45be-893e-24251b185686 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.506405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1c809f35-d457-45be-893e-24251b185686 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.570730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1c809f35-d457-45be-893e-24251b185686 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088683 May 25 16:22:26.572638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 379/741] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:22:26.590909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6e997392-36e0-4d7b-b355-efac3a60f439 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:26.592197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6e997392-36e0-4d7b-b355-efac3a60f439 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:26.618157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e997392-36e0-4d7b-b355-efac3a60f439 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.618936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e997392-36e0-4d7b-b355-efac3a60f439 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.625776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e997392-36e0-4d7b-b355-efac3a60f439 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.628499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e997392-36e0-4d7b-b355-efac3a60f439 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.727310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6e997392-36e0-4d7b-b355-efac3a60f439 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:26.752108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6e997392-36e0-4d7b-b355-efac3a60f439 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1607 microversion: 2.1 time: 0.165907 May 25 16:22:26.753023 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 380/742] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:26 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1607 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:26.803314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-eb36a44b-245c-4006-ae00-962777c05958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:26.810732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-eb36a44b-245c-4006-ae00-962777c05958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:26.835475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb36a44b-245c-4006-ae00-962777c05958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.837715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb36a44b-245c-4006-ae00-962777c05958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.838665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb36a44b-245c-4006-ae00-962777c05958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.839533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb36a44b-245c-4006-ae00-962777c05958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.869134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9713f023-0278-439f-8187-6f139374fd60 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:26.880646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-eb36a44b-245c-4006-ae00-962777c05958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.102340 May 25 16:22:26.881568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 381/743] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:26.884255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9713f023-0278-439f-8187-6f139374fd60 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.461535 May 25 16:22:26.885351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 363/744] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:26 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:26.886318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-099e2160-769d-443d-bb37-5a17ea4ecef8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:26.886928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-099e2160-769d-443d-bb37-5a17ea4ecef8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:26.893763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-099e2160-769d-443d-bb37-5a17ea4ecef8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.896524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-099e2160-769d-443d-bb37-5a17ea4ecef8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.900379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-099e2160-769d-443d-bb37-5a17ea4ecef8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:26.900830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-099e2160-769d-443d-bb37-5a17ea4ecef8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:26.976049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-099e2160-769d-443d-bb37-5a17ea4ecef8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:26.984012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-099e2160-769d-443d-bb37-5a17ea4ecef8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.101252 May 25 16:22:26.984667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 382/745] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:26 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:27.190416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-22f9b7d0-c806-475c-a67a-2d9673c59d8d req-caac6a38-169d-47f2-a7fb-e6bbb072f819 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5ce685f-a7e1-41d6-96f4-e5cc19cad00a", "name": "network-vif-deleted", "tag": "bb6d609e-7a8d-4258-adc8-66a6a5a24851"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:27.208412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-22f9b7d0-c806-475c-a67a-2d9673c59d8d req-caac6a38-169d-47f2-a7fb-e6bbb072f819 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.209991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-22f9b7d0-c806-475c-a67a-2d9673c59d8d req-caac6a38-169d-47f2-a7fb-e6bbb072f819 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.241262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-22f9b7d0-c806-475c-a67a-2d9673c59d8d req-caac6a38-169d-47f2-a7fb-e6bbb072f819 service nova] Creating event network-vif-deleted:bb6d609e-7a8d-4258-adc8-66a6a5a24851 for instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a on hv-cinder-84327 May 25 16:22:27.252329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-22f9b7d0-c806-475c-a67a-2d9673c59d8d req-caac6a38-169d-47f2-a7fb-e6bbb072f819 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065624 May 25 16:22:27.256761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 364/746] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:22:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:27.585658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f22f8b01-9bd9-4c26-82ec-3d015329ce90 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:27.586803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f22f8b01-9bd9-4c26-82ec-3d015329ce90 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:27.594616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f22f8b01-9bd9-4c26-82ec-3d015329ce90 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.600571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f22f8b01-9bd9-4c26-82ec-3d015329ce90 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.601683 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f22f8b01-9bd9-4c26-82ec-3d015329ce90 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.602332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f22f8b01-9bd9-4c26-82ec-3d015329ce90 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.658368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f22f8b01-9bd9-4c26-82ec-3d015329ce90 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.076652 May 25 16:22:27.659330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 383/747] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:27 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:27.770801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:27.771553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:27.777934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.778387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.779176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.779643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.782309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:27.811662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.812108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.812553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lazy-loading 'fault' on Instance uuid f5ce685f-a7e1-41d6-96f4-e5cc19cad00a {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:22:27.855028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:27.862975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-180133f9-d231-49d9-978c-6dc7a6d2e884 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 200 len: 1328 microversion: 2.1 time: 0.096955 May 25 16:22:27.863639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 365/748] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:27 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 1328 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:27.892758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bd95b026-1e6d-4b6f-8884-b95e15029b22 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:27.893683 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bd95b026-1e6d-4b6f-8884-b95e15029b22 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:27.899896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd95b026-1e6d-4b6f-8884-b95e15029b22 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.900426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd95b026-1e6d-4b6f-8884-b95e15029b22 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.901726 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd95b026-1e6d-4b6f-8884-b95e15029b22 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.902162 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd95b026-1e6d-4b6f-8884-b95e15029b22 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.906361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f2d73a18-4679-4a1f-940e-303afe75a6da tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:27.907090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f2d73a18-4679-4a1f-940e-303afe75a6da tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:27.914750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f2d73a18-4679-4a1f-940e-303afe75a6da tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.915718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f2d73a18-4679-4a1f-940e-303afe75a6da tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.916860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f2d73a18-4679-4a1f-940e-303afe75a6da tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:27.917402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f2d73a18-4679-4a1f-940e-303afe75a6da tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:27.986532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bd95b026-1e6d-4b6f-8884-b95e15029b22 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.097720 May 25 16:22:27.992321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 384/749] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:28.001937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c85a00aa-a6ae-4527-8ddf-3fe61a20fc4a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:28.003571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c85a00aa-a6ae-4527-8ddf-3fe61a20fc4a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:28.012909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c85a00aa-a6ae-4527-8ddf-3fe61a20fc4a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:28.013444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c85a00aa-a6ae-4527-8ddf-3fe61a20fc4a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:28.015130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c85a00aa-a6ae-4527-8ddf-3fe61a20fc4a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:28.016920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c85a00aa-a6ae-4527-8ddf-3fe61a20fc4a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:28.094047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c85a00aa-a6ae-4527-8ddf-3fe61a20fc4a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:28.101873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c85a00aa-a6ae-4527-8ddf-3fe61a20fc4a tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.104316 May 25 16:22:28.102876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 385/750] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:27 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:28.122141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f2d73a18-4679-4a1f-940e-303afe75a6da tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:28.130483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f2d73a18-4679-4a1f-940e-303afe75a6da tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.228093 May 25 16:22:28.131187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 366/751] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:27 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:28.674749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7ca49136-df7f-4a67-a3ad-8920ddf43ea5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:28.676310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7ca49136-df7f-4a67-a3ad-8920ddf43ea5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:28.686530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca49136-df7f-4a67-a3ad-8920ddf43ea5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:28.687328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca49136-df7f-4a67-a3ad-8920ddf43ea5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:28.688586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca49136-df7f-4a67-a3ad-8920ddf43ea5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:28.689307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca49136-df7f-4a67-a3ad-8920ddf43ea5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:28.721258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7ca49136-df7f-4a67-a3ad-8920ddf43ea5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053085 May 25 16:22:28.721811 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 386/752] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:28.884795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:28.885377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:28.892505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:28.892939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:28.893740 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:28.894174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:28.908150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] HTTP exception thrown: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Request method failure captured: May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a HTTP/1.1 May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaSt3NdfdTff4OH_doadoVd4ihuLN8fcTpnktVD_XWHHevyVxY4tmWr46X1fVz2TuvDT5fSQ1evXFm6-DfkMDQmPYTRLtjUiNx8nVTeCIRoDB7T5LAC8qRTpjGqBOay7ujWQTOdtPe6UQNGpwsHetNPMu5dXLWOI1Kk0Q9IbllFii8 May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:22:28.909811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeTestJSON-755332015 May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeTestJSON-755332015-project May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: e4e4d39c5ed249bc9dc097ba4c24b3d2 May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeTestJSON-755332015-project May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:22:28.911400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'f5ce685f-a7e1-41d6-96f4-e5cc19cad00a'} May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:22:28.912243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:22:28.913682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:22:28.914960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Returning 404 to user: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:22:28.914960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bf699aed-2f2d-444d-adc7-cd084fcfb9c1 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 404 len: 111 microversion: 2.1 time: 0.031652 May 25 16:22:28.914960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 367/753] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:28 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:22:28.920647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-008a3fdb-ceab-4cda-aca0-e049a387571e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '.version_select at 0x7f4a300401f0>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:28.940807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-008a3fdb-ceab-4cda-aca0-e049a387571e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1511807418" status: 202 len: 0 microversion: 2.1 time: 0.023888 May 25 16:22:28.941305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 387/754] 10.222.0.45 () {58 vars in 1220 bytes} [Wed May 25 16:22:28 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1511807418 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:22:29.003870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c3aa01f5-bf9a-4c52-86dd-698d0f818300 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:29.004861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c3aa01f5-bf9a-4c52-86dd-698d0f818300 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:29.011334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3aa01f5-bf9a-4c52-86dd-698d0f818300 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:29.011813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3aa01f5-bf9a-4c52-86dd-698d0f818300 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:29.012557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3aa01f5-bf9a-4c52-86dd-698d0f818300 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:29.012993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3aa01f5-bf9a-4c52-86dd-698d0f818300 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:29.045402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c3aa01f5-bf9a-4c52-86dd-698d0f818300 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045784 May 25 16:22:29.045971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 368/755] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:29.120407 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0b59b83f-b6bf-4d33-be3c-1496869e5e6c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:29.121102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0b59b83f-b6bf-4d33-be3c-1496869e5e6c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:29.127382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b59b83f-b6bf-4d33-be3c-1496869e5e6c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:29.127829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b59b83f-b6bf-4d33-be3c-1496869e5e6c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:29.128753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b59b83f-b6bf-4d33-be3c-1496869e5e6c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:29.129185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b59b83f-b6bf-4d33-be3c-1496869e5e6c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:29.148126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2bfdeb7a-cb59-4a2f-aaf4-5fe58a106410 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:29.148756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2bfdeb7a-cb59-4a2f-aaf4-5fe58a106410 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:29.155011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2bfdeb7a-cb59-4a2f-aaf4-5fe58a106410 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:29.155498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2bfdeb7a-cb59-4a2f-aaf4-5fe58a106410 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:29.156272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2bfdeb7a-cb59-4a2f-aaf4-5fe58a106410 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:29.156706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2bfdeb7a-cb59-4a2f-aaf4-5fe58a106410 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:29.255108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0b59b83f-b6bf-4d33-be3c-1496869e5e6c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:29.263548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0b59b83f-b6bf-4d33-be3c-1496869e5e6c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.147824 May 25 16:22:29.264140 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 388/756] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:29 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:29.385079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2bfdeb7a-cb59-4a2f-aaf4-5fe58a106410 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:29.393536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2bfdeb7a-cb59-4a2f-aaf4-5fe58a106410 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.249534 May 25 16:22:29.394212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 369/757] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:29 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:29.694588 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-449e637c-1cb8-4625-a78b-38b10ad0e4ba tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Action: 'create', calling method: .version_select at 0x7f4a237f9af0>, body: {"keypair": {"name": "tempest-keypair-883348019", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBDl1buVLC0PI1uH+xe0psIVwC1rDkiYStnW4pPDtuymRCo4Vx7OruDgPYAqCqFdZm6Kj96r8js8UP3rqk9YOE/YPEo+KQAyTNofPwaWDBZ8zs0ylbUa16sJZd/6bLNptw=="}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:29.708954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-449e637c-1cb8-4625-a78b-38b10ad0e4ba tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Getting quotas for project b36e59bfe72a49679b3b3657f5a65882. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:22:29.713163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-449e637c-1cb8-4625-a78b-38b10ad0e4ba tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Getting quotas for user e4e4d39c5ed249bc9dc097ba4c24b3d2 and project b36e59bfe72a49679b3b3657f5a65882. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:22:29.738049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-449e637c-1cb8-4625-a78b-38b10ad0e4ba tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.047400 May 25 16:22:29.738603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 389/758] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:22:29 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:29.842991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-07f5a65c-b40c-4345-87ed-c67934a0e465 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:29.844207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-07f5a65c-b40c-4345-87ed-c67934a0e465 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:29.851540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07f5a65c-b40c-4345-87ed-c67934a0e465 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:29.852064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07f5a65c-b40c-4345-87ed-c67934a0e465 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:29.852917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07f5a65c-b40c-4345-87ed-c67934a0e465 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:29.853356 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07f5a65c-b40c-4345-87ed-c67934a0e465 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:29.885599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-07f5a65c-b40c-4345-87ed-c67934a0e465 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.156002 May 25 16:22:29.886488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 370/759] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:29 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:30.057436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6049ca9a-32bc-4f35-ba65-323e225bd3fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:30.058399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6049ca9a-32bc-4f35-ba65-323e225bd3fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:30.065616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6049ca9a-32bc-4f35-ba65-323e225bd3fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:30.066058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6049ca9a-32bc-4f35-ba65-323e225bd3fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:30.066861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6049ca9a-32bc-4f35-ba65-323e225bd3fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:30.067348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6049ca9a-32bc-4f35-ba65-323e225bd3fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:30.098620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6049ca9a-32bc-4f35-ba65-323e225bd3fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045256 May 25 16:22:30.099175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 390/760] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:30 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:30.280634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-74e22d91-5b99-4670-8fe1-f4daa69d1370 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:30.281339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-74e22d91-5b99-4670-8fe1-f4daa69d1370 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:30.288288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74e22d91-5b99-4670-8fe1-f4daa69d1370 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:30.288878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74e22d91-5b99-4670-8fe1-f4daa69d1370 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:30.289744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74e22d91-5b99-4670-8fe1-f4daa69d1370 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:30.290186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74e22d91-5b99-4670-8fe1-f4daa69d1370 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:30.372914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-74e22d91-5b99-4670-8fe1-f4daa69d1370 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:30.380481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-74e22d91-5b99-4670-8fe1-f4daa69d1370 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.103813 May 25 16:22:30.381027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 371/761] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:30 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:30.411225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-77ba5aa4-7716-419a-b938-69a27ef34eb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:30.411840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-77ba5aa4-7716-419a-b938-69a27ef34eb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:30.418377 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77ba5aa4-7716-419a-b938-69a27ef34eb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:30.418832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77ba5aa4-7716-419a-b938-69a27ef34eb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:30.419620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77ba5aa4-7716-419a-b938-69a27ef34eb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:30.420150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-77ba5aa4-7716-419a-b938-69a27ef34eb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:30.597810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-77ba5aa4-7716-419a-b938-69a27ef34eb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:30.608564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-77ba5aa4-7716-419a-b938-69a27ef34eb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.201382 May 25 16:22:30.609206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 391/762] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:30 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:30.897764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f752142f-ed68-421a-9dcd-40b81112ce22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:30.898913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f752142f-ed68-421a-9dcd-40b81112ce22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:30.905195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f752142f-ed68-421a-9dcd-40b81112ce22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:30.905640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f752142f-ed68-421a-9dcd-40b81112ce22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:30.906527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f752142f-ed68-421a-9dcd-40b81112ce22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:30.906956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f752142f-ed68-421a-9dcd-40b81112ce22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:30.941677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f752142f-ed68-421a-9dcd-40b81112ce22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048362 May 25 16:22:30.942478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 372/763] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:30 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:31.109993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fb918266-ef15-403d-80b6-47c7c3573bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:31.110954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fb918266-ef15-403d-80b6-47c7c3573bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:31.117567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fb918266-ef15-403d-80b6-47c7c3573bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.118011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fb918266-ef15-403d-80b6-47c7c3573bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.123083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fb918266-ef15-403d-80b6-47c7c3573bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.123083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fb918266-ef15-403d-80b6-47c7c3573bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.128854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-151730091", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "28311134-63e6-4448-9485-57f8501da8ed"}], "security_groups": [{"name": "tempest-securitygroup--1420048694"}], "key_name": "tempest-keypair-883348019", "user_data": "}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:31.150495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fb918266-ef15-403d-80b6-47c7c3573bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044352 May 25 16:22:31.151218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 392/764] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:31 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:31.232074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] validate_networks() for [('28311134-63e6-4448-9485-57f8501da8ed', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:22:31.397377 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ca92571e-a4ab-4b96-916d-d7d879fa5c47 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:31.398209 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ca92571e-a4ab-4b96-916d-d7d879fa5c47 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:31.406989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca92571e-a4ab-4b96-916d-d7d879fa5c47 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.407382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca92571e-a4ab-4b96-916d-d7d879fa5c47 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.408154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca92571e-a4ab-4b96-916d-d7d879fa5c47 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.408630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca92571e-a4ab-4b96-916d-d7d879fa5c47 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.474890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ca92571e-a4ab-4b96-916d-d7d879fa5c47 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:31.482788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ca92571e-a4ab-4b96-916d-d7d879fa5c47 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.089286 May 25 16:22:31.483560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 393/765] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:31 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:31.661968 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:22:31.662376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:22:31.662872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:22:31.663409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:22:31.663819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:22:31.673064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.673706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.674503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.674937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.691515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Getting quotas for project b36e59bfe72a49679b3b3657f5a65882. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:22:31.695556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Getting quotas for user e4e4d39c5ed249bc9dc097ba4c24b3d2 and project b36e59bfe72a49679b3b3657f5a65882. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:22:31.706423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:22:31.708766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:22:31.729421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-70db85cf-b276-4470-a214-78521748de84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:31.730109 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-70db85cf-b276-4470-a214-78521748de84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:31.736472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70db85cf-b276-4470-a214-78521748de84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.736911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70db85cf-b276-4470-a214-78521748de84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.737622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70db85cf-b276-4470-a214-78521748de84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.738307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70db85cf-b276-4470-a214-78521748de84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.755032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:31.758778 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-775b484c-65e9-4ad5-b352-410364918023 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.634178 May 25 16:22:31.763655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 373/766] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:22:31 2022] POST /compute/v2.1/servers => generated 402 bytes in 640 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:22:31.770108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7ae4f67d-8f6a-4d75-a132-41aa21162b48 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:31.770875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7ae4f67d-8f6a-4d75-a132-41aa21162b48 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:31.843041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7ae4f67d-8f6a-4d75-a132-41aa21162b48 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:31.851224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7ae4f67d-8f6a-4d75-a132-41aa21162b48 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1236 microversion: 2.1 time: 0.085164 May 25 16:22:31.855238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 374/767] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:31 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1236 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:31.906501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-70db85cf-b276-4470-a214-78521748de84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:31.915125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-70db85cf-b276-4470-a214-78521748de84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.292217 May 25 16:22:31.915976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 394/768] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:31 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:31.953909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e0446901-3d24-471e-ab98-79cd2fe9d6a0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:31.955040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e0446901-3d24-471e-ab98-79cd2fe9d6a0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:31.962087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0446901-3d24-471e-ab98-79cd2fe9d6a0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.962815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0446901-3d24-471e-ab98-79cd2fe9d6a0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.963739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0446901-3d24-471e-ab98-79cd2fe9d6a0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:31.964222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0446901-3d24-471e-ab98-79cd2fe9d6a0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:31.994584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e0446901-3d24-471e-ab98-79cd2fe9d6a0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044659 May 25 16:22:31.995179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 375/769] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:31 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:32.162296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0d522c22-7542-4ddc-ae50-3493eac4a9c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:32.163540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0d522c22-7542-4ddc-ae50-3493eac4a9c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:32.171078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d522c22-7542-4ddc-ae50-3493eac4a9c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:32.171540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d522c22-7542-4ddc-ae50-3493eac4a9c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:32.172426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d522c22-7542-4ddc-ae50-3493eac4a9c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:32.172845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d522c22-7542-4ddc-ae50-3493eac4a9c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:32.205707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0d522c22-7542-4ddc-ae50-3493eac4a9c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047243 May 25 16:22:32.206378 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 395/770] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:32 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:22:32.504048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f14bcbfa-861f-42b1-82cc-7b2127966022 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:32.504949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f14bcbfa-861f-42b1-82cc-7b2127966022 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:32.512665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f14bcbfa-861f-42b1-82cc-7b2127966022 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:32.513108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f14bcbfa-861f-42b1-82cc-7b2127966022 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:32.513829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f14bcbfa-861f-42b1-82cc-7b2127966022 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:32.514257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f14bcbfa-861f-42b1-82cc-7b2127966022 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:32.580022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f14bcbfa-861f-42b1-82cc-7b2127966022 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:32.587975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f14bcbfa-861f-42b1-82cc-7b2127966022 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.090882 May 25 16:22:32.588533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 376/771] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:32 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:32.868256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c0d84c98-fa07-4c79-9dd5-ea242f395509 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:32.868853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c0d84c98-fa07-4c79-9dd5-ea242f395509 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:32.875528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c0d84c98-fa07-4c79-9dd5-ea242f395509 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:32.875982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c0d84c98-fa07-4c79-9dd5-ea242f395509 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:32.876750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c0d84c98-fa07-4c79-9dd5-ea242f395509 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:32.877184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c0d84c98-fa07-4c79-9dd5-ea242f395509 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:32.934002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ced11ccc-40d5-4a76-a46d-166bd467ab30 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:32.934701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ced11ccc-40d5-4a76-a46d-166bd467ab30 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:32.945530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ced11ccc-40d5-4a76-a46d-166bd467ab30 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:32.946091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ced11ccc-40d5-4a76-a46d-166bd467ab30 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:32.946936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ced11ccc-40d5-4a76-a46d-166bd467ab30 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:32.950733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ced11ccc-40d5-4a76-a46d-166bd467ab30 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:32.973086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c0d84c98-fa07-4c79-9dd5-ea242f395509 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:32.981647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c0d84c98-fa07-4c79-9dd5-ea242f395509 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1306 microversion: 2.1 time: 0.117507 May 25 16:22:32.982220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 396/772] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:32 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1306 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:33.005868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1583caea-2688-4d0f-b283-f0488a52131a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:33.006963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1583caea-2688-4d0f-b283-f0488a52131a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:33.013169 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1583caea-2688-4d0f-b283-f0488a52131a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:33.013707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1583caea-2688-4d0f-b283-f0488a52131a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:33.014528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1583caea-2688-4d0f-b283-f0488a52131a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:33.015038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1583caea-2688-4d0f-b283-f0488a52131a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:33.044639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1583caea-2688-4d0f-b283-f0488a52131a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042442 May 25 16:22:33.045419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 397/773] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:33 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:22:33.090013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ced11ccc-40d5-4a76-a46d-166bd467ab30 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:33.098214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ced11ccc-40d5-4a76-a46d-166bd467ab30 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.169614 May 25 16:22:33.098789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 377/774] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:32 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:33.219079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bd27a516-2453-4846-badf-6198b1cd9f61 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:33.220041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bd27a516-2453-4846-badf-6198b1cd9f61 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:33.228053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd27a516-2453-4846-badf-6198b1cd9f61 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:33.228567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd27a516-2453-4846-badf-6198b1cd9f61 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:33.229452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd27a516-2453-4846-badf-6198b1cd9f61 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:33.229884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd27a516-2453-4846-badf-6198b1cd9f61 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:33.261540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bd27a516-2453-4846-badf-6198b1cd9f61 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047385 May 25 16:22:33.262556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 398/775] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:33 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:33.604689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a4f31f42-616a-41bf-8add-3efe6ca7fb7c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:33.605320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a4f31f42-616a-41bf-8add-3efe6ca7fb7c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:33.612409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4f31f42-616a-41bf-8add-3efe6ca7fb7c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:33.613216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4f31f42-616a-41bf-8add-3efe6ca7fb7c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:33.614186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4f31f42-616a-41bf-8add-3efe6ca7fb7c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:33.614866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4f31f42-616a-41bf-8add-3efe6ca7fb7c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:33.681064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a4f31f42-616a-41bf-8add-3efe6ca7fb7c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:33.689548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a4f31f42-616a-41bf-8add-3efe6ca7fb7c tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.088809 May 25 16:22:33.699209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 378/776] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:33 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:33.999378 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-262b77a2-8d6c-4297-b0c1-724927756b1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:33.999991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-262b77a2-8d6c-4297-b0c1-724927756b1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:34.006205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-262b77a2-8d6c-4297-b0c1-724927756b1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.006653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-262b77a2-8d6c-4297-b0c1-724927756b1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.007668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-262b77a2-8d6c-4297-b0c1-724927756b1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.008126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-262b77a2-8d6c-4297-b0c1-724927756b1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.056619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f904aa47-90ee-4203-bf72-dde108897882 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:34.057608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f904aa47-90ee-4203-bf72-dde108897882 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:34.065992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f904aa47-90ee-4203-bf72-dde108897882 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.066866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f904aa47-90ee-4203-bf72-dde108897882 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.067905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f904aa47-90ee-4203-bf72-dde108897882 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.068616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f904aa47-90ee-4203-bf72-dde108897882 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.108244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f904aa47-90ee-4203-bf72-dde108897882 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056698 May 25 16:22:34.113022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 379/777] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:34 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:34.119608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-af723deb-663a-4d98-bcf9-f696595616f3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:34.121172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-af723deb-663a-4d98-bcf9-f696595616f3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:34.130284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-af723deb-663a-4d98-bcf9-f696595616f3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.130992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-af723deb-663a-4d98-bcf9-f696595616f3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.132396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-af723deb-663a-4d98-bcf9-f696595616f3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.133419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-af723deb-663a-4d98-bcf9-f696595616f3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.171110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-262b77a2-8d6c-4297-b0c1-724927756b1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:34.179756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-262b77a2-8d6c-4297-b0c1-724927756b1a tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1362 microversion: 2.1 time: 0.184548 May 25 16:22:34.180571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 399/778] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:33 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1362 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:34.274650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-85356db1-46e3-43e8-ae43-246dfc8b390c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:34.275890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-85356db1-46e3-43e8-ae43-246dfc8b390c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:34.297341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85356db1-46e3-43e8-ae43-246dfc8b390c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.301532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-af723deb-663a-4d98-bcf9-f696595616f3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:34.306428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85356db1-46e3-43e8-ae43-246dfc8b390c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.312033 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85356db1-46e3-43e8-ae43-246dfc8b390c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.313237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85356db1-46e3-43e8-ae43-246dfc8b390c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.314977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-af723deb-663a-4d98-bcf9-f696595616f3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.200270 May 25 16:22:34.326931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 380/779] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:34 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:34.335142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-4eca3bab-78df-4da0-be7a-3ae6c4f945c5 req-93a4ec12-cc38-419d-846e-d45c068c4fd9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c98878d3-b9b6-4688-abea-b14c2150fb4a", "tag": "9beaea24-9678-4ef5-8aee-b08193e580ba"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:34.345011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-4eca3bab-78df-4da0-be7a-3ae6c4f945c5 req-93a4ec12-cc38-419d-846e-d45c068c4fd9 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.345741 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-4eca3bab-78df-4da0-be7a-3ae6c4f945c5 req-93a4ec12-cc38-419d-846e-d45c068c4fd9 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.363292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-4eca3bab-78df-4da0-be7a-3ae6c4f945c5 req-93a4ec12-cc38-419d-846e-d45c068c4fd9 service nova] Creating event network-changed:9beaea24-9678-4ef5-8aee-b08193e580ba for instance c98878d3-b9b6-4688-abea-b14c2150fb4a on hv-cinder-84327 May 25 16:22:34.366772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-85356db1-46e3-43e8-ae43-246dfc8b390c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.096923 May 25 16:22:34.368131 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 400/780] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:34 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:34.368732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-4eca3bab-78df-4da0-be7a-3ae6c4f945c5 req-93a4ec12-cc38-419d-846e-d45c068c4fd9 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040329 May 25 16:22:34.369517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 381/781] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:22:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:34.706122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ac6d413d-1141-4059-8f76-aa74778ba081 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:34.706734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ac6d413d-1141-4059-8f76-aa74778ba081 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:34.713315 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac6d413d-1141-4059-8f76-aa74778ba081 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.713901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac6d413d-1141-4059-8f76-aa74778ba081 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.714706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac6d413d-1141-4059-8f76-aa74778ba081 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:34.715137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac6d413d-1141-4059-8f76-aa74778ba081 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:34.776251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ac6d413d-1141-4059-8f76-aa74778ba081 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:34.784411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ac6d413d-1141-4059-8f76-aa74778ba081 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.082239 May 25 16:22:34.784987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 401/782] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:34 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:35.121215 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3e6d595a-035b-4e81-aaec-186134a6d68d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:35.122237 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3e6d595a-035b-4e81-aaec-186134a6d68d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:35.129606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e6d595a-035b-4e81-aaec-186134a6d68d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.130060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e6d595a-035b-4e81-aaec-186134a6d68d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.130824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e6d595a-035b-4e81-aaec-186134a6d68d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.131296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e6d595a-035b-4e81-aaec-186134a6d68d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.161075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3e6d595a-035b-4e81-aaec-186134a6d68d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044312 May 25 16:22:35.161625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 382/783] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:35 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:35.197402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7d2c7362-9c29-4bca-abee-92dd6f3c3c37 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:35.198004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7d2c7362-9c29-4bca-abee-92dd6f3c3c37 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:35.204501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d2c7362-9c29-4bca-abee-92dd6f3c3c37 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.204945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d2c7362-9c29-4bca-abee-92dd6f3c3c37 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.205694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d2c7362-9c29-4bca-abee-92dd6f3c3c37 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.206120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d2c7362-9c29-4bca-abee-92dd6f3c3c37 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.338700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7d2c7362-9c29-4bca-abee-92dd6f3c3c37 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:35.341910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-649b7548-8e6c-49aa-a14a-102a8cfd8c0b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:35.342527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-649b7548-8e6c-49aa-a14a-102a8cfd8c0b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:35.346582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7d2c7362-9c29-4bca-abee-92dd6f3c3c37 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1362 microversion: 2.1 time: 0.153108 May 25 16:22:35.347235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 402/784] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:35 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:35.357580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-649b7548-8e6c-49aa-a14a-102a8cfd8c0b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.358025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-649b7548-8e6c-49aa-a14a-102a8cfd8c0b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.358792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-649b7548-8e6c-49aa-a14a-102a8cfd8c0b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.359281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-649b7548-8e6c-49aa-a14a-102a8cfd8c0b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.378698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d2a1a7c7-950d-4ba7-8ea5-b76d97be9d06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:35.379760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d2a1a7c7-950d-4ba7-8ea5-b76d97be9d06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:35.386629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2a1a7c7-950d-4ba7-8ea5-b76d97be9d06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.387112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2a1a7c7-950d-4ba7-8ea5-b76d97be9d06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.388012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2a1a7c7-950d-4ba7-8ea5-b76d97be9d06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.388487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2a1a7c7-950d-4ba7-8ea5-b76d97be9d06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.421756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d2a1a7c7-950d-4ba7-8ea5-b76d97be9d06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047613 May 25 16:22:35.422332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 403/785] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:35 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:22:35.491732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-649b7548-8e6c-49aa-a14a-102a8cfd8c0b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:35.500105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-649b7548-8e6c-49aa-a14a-102a8cfd8c0b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.162031 May 25 16:22:35.500682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 383/786] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:35 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:35.802126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-709b0a17-92dd-47c5-a67a-1efa141f5085 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:35.802736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-709b0a17-92dd-47c5-a67a-1efa141f5085 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:35.809124 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-709b0a17-92dd-47c5-a67a-1efa141f5085 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.809673 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-709b0a17-92dd-47c5-a67a-1efa141f5085 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.810718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-709b0a17-92dd-47c5-a67a-1efa141f5085 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:35.811323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-709b0a17-92dd-47c5-a67a-1efa141f5085 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:35.875541 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-709b0a17-92dd-47c5-a67a-1efa141f5085 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:35.883466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-709b0a17-92dd-47c5-a67a-1efa141f5085 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.085543 May 25 16:22:35.884017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 404/787] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:35 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:36.173258 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6bc65338-9ad1-40e6-be6f-10eebebe130d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:36.174853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6bc65338-9ad1-40e6-be6f-10eebebe130d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:36.181627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6bc65338-9ad1-40e6-be6f-10eebebe130d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.182070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6bc65338-9ad1-40e6-be6f-10eebebe130d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.182823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6bc65338-9ad1-40e6-be6f-10eebebe130d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.183379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6bc65338-9ad1-40e6-be6f-10eebebe130d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.198108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:36.215633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6bc65338-9ad1-40e6-be6f-10eebebe130d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046294 May 25 16:22:36.216204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 384/788] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:36 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:36.368176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-61570201-1602-4b9d-afb9-b788b49f9c0f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:36.369177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-61570201-1602-4b9d-afb9-b788b49f9c0f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:36.379558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-61570201-1602-4b9d-afb9-b788b49f9c0f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.380391 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-61570201-1602-4b9d-afb9-b788b49f9c0f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.381658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-61570201-1602-4b9d-afb9-b788b49f9c0f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.382373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-61570201-1602-4b9d-afb9-b788b49f9c0f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.433742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dd7d94d6-dcad-4132-a971-9a7236fdac45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:36.434669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dd7d94d6-dcad-4132-a971-9a7236fdac45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:36.441184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd7d94d6-dcad-4132-a971-9a7236fdac45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.441629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd7d94d6-dcad-4132-a971-9a7236fdac45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.442396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd7d94d6-dcad-4132-a971-9a7236fdac45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.442825 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd7d94d6-dcad-4132-a971-9a7236fdac45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.472170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dd7d94d6-dcad-4132-a971-9a7236fdac45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042411 May 25 16:22:36.472698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 385/789] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:36 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:36.516971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-13c9fbb0-9fe9-464f-80b1-b1d9b3f1fbe8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:36.517580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-13c9fbb0-9fe9-464f-80b1-b1d9b3f1fbe8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:36.524079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13c9fbb0-9fe9-464f-80b1-b1d9b3f1fbe8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.524610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13c9fbb0-9fe9-464f-80b1-b1d9b3f1fbe8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.525463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-61570201-1602-4b9d-afb9-b788b49f9c0f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:36.525577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13c9fbb0-9fe9-464f-80b1-b1d9b3f1fbe8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.526001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13c9fbb0-9fe9-464f-80b1-b1d9b3f1fbe8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.532801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-61570201-1602-4b9d-afb9-b788b49f9c0f tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1362 microversion: 2.1 time: 0.171268 May 25 16:22:36.533591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 405/790] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:36 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1362 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:36.660513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-13c9fbb0-9fe9-464f-80b1-b1d9b3f1fbe8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:36.669113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-13c9fbb0-9fe9-464f-80b1-b1d9b3f1fbe8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.155965 May 25 16:22:36.669668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 386/791] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:36 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:36.903247 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5e4a4650-e7d9-4230-a51d-4eb3e25b1a6d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:36.903843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5e4a4650-e7d9-4230-a51d-4eb3e25b1a6d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:36.911028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e4a4650-e7d9-4230-a51d-4eb3e25b1a6d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.911510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e4a4650-e7d9-4230-a51d-4eb3e25b1a6d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.912388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e4a4650-e7d9-4230-a51d-4eb3e25b1a6d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:36.912855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e4a4650-e7d9-4230-a51d-4eb3e25b1a6d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:36.979006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5e4a4650-e7d9-4230-a51d-4eb3e25b1a6d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:36.991873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5e4a4650-e7d9-4230-a51d-4eb3e25b1a6d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.092775 May 25 16:22:36.992768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 406/792] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:36 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:37.231923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-526cb3df-a74f-4f80-be50-27e57ed9deed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:37.233275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-526cb3df-a74f-4f80-be50-27e57ed9deed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:37.244307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-526cb3df-a74f-4f80-be50-27e57ed9deed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:37.245046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-526cb3df-a74f-4f80-be50-27e57ed9deed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:37.246241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-526cb3df-a74f-4f80-be50-27e57ed9deed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:37.246998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-526cb3df-a74f-4f80-be50-27e57ed9deed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:37.280558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-526cb3df-a74f-4f80-be50-27e57ed9deed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055781 May 25 16:22:37.281107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 387/793] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:37 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:37.489210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-732c17b8-bb2f-4b92-8fcc-0ac7110555c6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:37.490617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-732c17b8-bb2f-4b92-8fcc-0ac7110555c6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:37.497718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-732c17b8-bb2f-4b92-8fcc-0ac7110555c6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:37.498168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-732c17b8-bb2f-4b92-8fcc-0ac7110555c6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:37.498924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-732c17b8-bb2f-4b92-8fcc-0ac7110555c6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:37.499387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-732c17b8-bb2f-4b92-8fcc-0ac7110555c6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:37.529986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-732c17b8-bb2f-4b92-8fcc-0ac7110555c6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048257 May 25 16:22:37.530535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 407/794] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:37 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:37.551738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-75855bfb-9def-4c70-bd58-0987be31fc6c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:37.552718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-75855bfb-9def-4c70-bd58-0987be31fc6c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:37.559375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75855bfb-9def-4c70-bd58-0987be31fc6c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:37.559817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75855bfb-9def-4c70-bd58-0987be31fc6c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:37.560567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75855bfb-9def-4c70-bd58-0987be31fc6c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:37.560992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75855bfb-9def-4c70-bd58-0987be31fc6c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:37.686432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a1a06320-29e6-4b45-a720-651e27f5ff41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:37.687067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a1a06320-29e6-4b45-a720-651e27f5ff41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:37.692919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-75855bfb-9def-4c70-bd58-0987be31fc6c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:37.693413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1a06320-29e6-4b45-a720-651e27f5ff41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:37.693936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1a06320-29e6-4b45-a720-651e27f5ff41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:37.694851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1a06320-29e6-4b45-a720-651e27f5ff41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:37.695316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1a06320-29e6-4b45-a720-651e27f5ff41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:37.701016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-75855bfb-9def-4c70-bd58-0987be31fc6c tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1362 microversion: 2.1 time: 0.154935 May 25 16:22:37.701576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 388/795] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:37 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:37.820716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a1a06320-29e6-4b45-a720-651e27f5ff41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:37.828879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a1a06320-29e6-4b45-a720-651e27f5ff41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.146786 May 25 16:22:37.829479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 408/796] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:37 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:38.019128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7d64ebe7-2638-47d1-9114-fdfe875cd992 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:38.020278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7d64ebe7-2638-47d1-9114-fdfe875cd992 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:38.022944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:38.028485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d64ebe7-2638-47d1-9114-fdfe875cd992 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.028940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d64ebe7-2638-47d1-9114-fdfe875cd992 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.029710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d64ebe7-2638-47d1-9114-fdfe875cd992 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.030146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d64ebe7-2638-47d1-9114-fdfe875cd992 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.100253 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7d64ebe7-2638-47d1-9114-fdfe875cd992 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:38.108499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7d64ebe7-2638-47d1-9114-fdfe875cd992 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.097003 May 25 16:22:38.109070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 389/797] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:38 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:38.294505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ebe08b29-6297-4bc8-bbc9-3880f620845a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:38.295462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ebe08b29-6297-4bc8-bbc9-3880f620845a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:38.301937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ebe08b29-6297-4bc8-bbc9-3880f620845a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.302382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ebe08b29-6297-4bc8-bbc9-3880f620845a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.303128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ebe08b29-6297-4bc8-bbc9-3880f620845a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.303592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ebe08b29-6297-4bc8-bbc9-3880f620845a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.333252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ebe08b29-6297-4bc8-bbc9-3880f620845a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043034 May 25 16:22:38.333821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 409/798] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:38 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:38.547283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f06571dd-1aee-4253-9282-471756909344 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:38.548760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f06571dd-1aee-4253-9282-471756909344 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:38.559963 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f06571dd-1aee-4253-9282-471756909344 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.560804 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f06571dd-1aee-4253-9282-471756909344 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.562229 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f06571dd-1aee-4253-9282-471756909344 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.563076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f06571dd-1aee-4253-9282-471756909344 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.596460 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f06571dd-1aee-4253-9282-471756909344 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056536 May 25 16:22:38.600036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 390/799] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:38 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:38.716369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-a34b11dc-d7a2-494f-a19f-7824db99ca89 req-d4b6a212-aede-4851-b12b-42d92fd95223 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c98878d3-b9b6-4688-abea-b14c2150fb4a", "name": "network-vif-plugged", "status": "completed", "tag": "9beaea24-9678-4ef5-8aee-b08193e580ba"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:38.718469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-31695ac8-8716-4b22-82e8-d7de84f14e4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:38.719183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-31695ac8-8716-4b22-82e8-d7de84f14e4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:38.724976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-a34b11dc-d7a2-494f-a19f-7824db99ca89 req-d4b6a212-aede-4851-b12b-42d92fd95223 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.725465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-a34b11dc-d7a2-494f-a19f-7824db99ca89 req-d4b6a212-aede-4851-b12b-42d92fd95223 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.727340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31695ac8-8716-4b22-82e8-d7de84f14e4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.727791 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31695ac8-8716-4b22-82e8-d7de84f14e4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.729195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31695ac8-8716-4b22-82e8-d7de84f14e4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.730615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31695ac8-8716-4b22-82e8-d7de84f14e4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.739740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-a34b11dc-d7a2-494f-a19f-7824db99ca89 req-d4b6a212-aede-4851-b12b-42d92fd95223 service nova] Creating event network-vif-plugged:9beaea24-9678-4ef5-8aee-b08193e580ba for instance c98878d3-b9b6-4688-abea-b14c2150fb4a on hv-cinder-84327 May 25 16:22:38.745165 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-a34b11dc-d7a2-494f-a19f-7824db99ca89 req-d4b6a212-aede-4851-b12b-42d92fd95223 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032920 May 25 16:22:38.745881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 410/800] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:22:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:38.847278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-119f1124-dc9f-432b-98d0-4a8ff9eedd65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:38.848070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-119f1124-dc9f-432b-98d0-4a8ff9eedd65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:38.854535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-119f1124-dc9f-432b-98d0-4a8ff9eedd65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.855070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-119f1124-dc9f-432b-98d0-4a8ff9eedd65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.856077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-119f1124-dc9f-432b-98d0-4a8ff9eedd65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:38.856575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-119f1124-dc9f-432b-98d0-4a8ff9eedd65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:38.870288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-31695ac8-8716-4b22-82e8-d7de84f14e4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:38.878154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-31695ac8-8716-4b22-82e8-d7de84f14e4d tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1362 microversion: 2.1 time: 0.164161 May 25 16:22:38.878729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 391/801] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:38 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1362 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:38.984969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-119f1124-dc9f-432b-98d0-4a8ff9eedd65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:38.987302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:38.994167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-119f1124-dc9f-432b-98d0-4a8ff9eedd65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.151825 May 25 16:22:38.994733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 411/802] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:38 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:39.129505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d25ad0ba-68c1-4ab8-b203-db00e3be2926 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:39.130643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d25ad0ba-68c1-4ab8-b203-db00e3be2926 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:39.140905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d25ad0ba-68c1-4ab8-b203-db00e3be2926 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:39.141660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d25ad0ba-68c1-4ab8-b203-db00e3be2926 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:39.142905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d25ad0ba-68c1-4ab8-b203-db00e3be2926 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:39.143689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d25ad0ba-68c1-4ab8-b203-db00e3be2926 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:39.216066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d25ad0ba-68c1-4ab8-b203-db00e3be2926 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:39.223792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d25ad0ba-68c1-4ab8-b203-db00e3be2926 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.100815 May 25 16:22:39.224376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 392/803] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:39 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:39.349803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-04cda76e-6767-457a-a8d7-09f4be5f541b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:39.351040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-04cda76e-6767-457a-a8d7-09f4be5f541b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:39.357445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-04cda76e-6767-457a-a8d7-09f4be5f541b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:39.357892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-04cda76e-6767-457a-a8d7-09f4be5f541b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:39.358653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-04cda76e-6767-457a-a8d7-09f4be5f541b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:39.359079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-04cda76e-6767-457a-a8d7-09f4be5f541b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:39.388735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-04cda76e-6767-457a-a8d7-09f4be5f541b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046006 May 25 16:22:39.389260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 412/804] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:39 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:39.614037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-23f5038f-6681-45a1-947a-f1af3deb1f3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:39.615516 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-23f5038f-6681-45a1-947a-f1af3deb1f3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:39.622732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23f5038f-6681-45a1-947a-f1af3deb1f3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:39.623183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23f5038f-6681-45a1-947a-f1af3deb1f3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:39.624119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23f5038f-6681-45a1-947a-f1af3deb1f3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:39.624554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23f5038f-6681-45a1-947a-f1af3deb1f3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:39.655095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-23f5038f-6681-45a1-947a-f1af3deb1f3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048016 May 25 16:22:39.655676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 393/805] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:39 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:39.899195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ee9c781c-a710-455f-aa16-94bab9af001e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:39.899750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ee9c781c-a710-455f-aa16-94bab9af001e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:39.902167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=4 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:39.902877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:39.910076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee9c781c-a710-455f-aa16-94bab9af001e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:39.910530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee9c781c-a710-455f-aa16-94bab9af001e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:39.911385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee9c781c-a710-455f-aa16-94bab9af001e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:39.911845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee9c781c-a710-455f-aa16-94bab9af001e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:40.011058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e22b99d0-84c5-4abf-8eb0-b627eff5879b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:40.011700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e22b99d0-84c5-4abf-8eb0-b627eff5879b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:40.018149 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e22b99d0-84c5-4abf-8eb0-b627eff5879b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:40.018604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e22b99d0-84c5-4abf-8eb0-b627eff5879b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:40.019434 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e22b99d0-84c5-4abf-8eb0-b627eff5879b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:40.019876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e22b99d0-84c5-4abf-8eb0-b627eff5879b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:40.045146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ee9c781c-a710-455f-aa16-94bab9af001e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:40.052923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ee9c781c-a710-455f-aa16-94bab9af001e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1362 microversion: 2.1 time: 0.160003 May 25 16:22:40.053590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 413/806] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:39 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1362 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:40.148566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e22b99d0-84c5-4abf-8eb0-b627eff5879b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:40.159898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e22b99d0-84c5-4abf-8eb0-b627eff5879b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.152792 May 25 16:22:40.160481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 394/807] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:40 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:40.249563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a8264df9-713a-4fd5-a93f-0658a20953c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:40.250546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a8264df9-713a-4fd5-a93f-0658a20953c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:40.259422 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8264df9-713a-4fd5-a93f-0658a20953c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:40.259878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8264df9-713a-4fd5-a93f-0658a20953c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:40.260628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8264df9-713a-4fd5-a93f-0658a20953c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:40.261054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8264df9-713a-4fd5-a93f-0658a20953c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:40.324906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a8264df9-713a-4fd5-a93f-0658a20953c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:40.332559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a8264df9-713a-4fd5-a93f-0658a20953c8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.089462 May 25 16:22:40.333120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 414/808] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:40 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:40.404713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a1f45074-3e18-4c98-b440-c4c02ef414fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:40.406275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a1f45074-3e18-4c98-b440-c4c02ef414fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:40.415880 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1f45074-3e18-4c98-b440-c4c02ef414fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:40.416474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1f45074-3e18-4c98-b440-c4c02ef414fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:40.417623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1f45074-3e18-4c98-b440-c4c02ef414fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:40.418319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1f45074-3e18-4c98-b440-c4c02ef414fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:40.451549 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a1f45074-3e18-4c98-b440-c4c02ef414fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052972 May 25 16:22:40.452109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 395/809] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:40 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:40.667299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a152ea1c-43fb-4b95-9c9b-0c3154745218 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:40.668160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a152ea1c-43fb-4b95-9c9b-0c3154745218 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:40.674563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a152ea1c-43fb-4b95-9c9b-0c3154745218 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:40.675017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a152ea1c-43fb-4b95-9c9b-0c3154745218 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:40.675870 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a152ea1c-43fb-4b95-9c9b-0c3154745218 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:40.676309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a152ea1c-43fb-4b95-9c9b-0c3154745218 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:40.705795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a152ea1c-43fb-4b95-9c9b-0c3154745218 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042947 May 25 16:22:40.706394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 415/810] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:40 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:41.074818 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3b3a9285-3f12-4543-9f6f-897dbedc7490 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:41.075595 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3b3a9285-3f12-4543-9f6f-897dbedc7490 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:41.086122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b3a9285-3f12-4543-9f6f-897dbedc7490 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.086573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b3a9285-3f12-4543-9f6f-897dbedc7490 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.087353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b3a9285-3f12-4543-9f6f-897dbedc7490 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.087789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b3a9285-3f12-4543-9f6f-897dbedc7490 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.177987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b17c2319-54e7-4f95-9eb8-ced903e35961 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:41.178666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b17c2319-54e7-4f95-9eb8-ced903e35961 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:41.186051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b17c2319-54e7-4f95-9eb8-ced903e35961 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.186586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b17c2319-54e7-4f95-9eb8-ced903e35961 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.187449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b17c2319-54e7-4f95-9eb8-ced903e35961 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.187980 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b17c2319-54e7-4f95-9eb8-ced903e35961 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.222642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3b3a9285-3f12-4543-9f6f-897dbedc7490 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:41.231352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3b3a9285-3f12-4543-9f6f-897dbedc7490 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1548 microversion: 2.1 time: 0.163230 May 25 16:22:41.231614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 396/811] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:41 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1548 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:41.250409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5dbafb8c-b10b-4845-b2c8-430eec2a4472 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:41.250868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5dbafb8c-b10b-4845-b2c8-430eec2a4472 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:41.257448 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5dbafb8c-b10b-4845-b2c8-430eec2a4472 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.257898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5dbafb8c-b10b-4845-b2c8-430eec2a4472 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.258661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5dbafb8c-b10b-4845-b2c8-430eec2a4472 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.259102 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5dbafb8c-b10b-4845-b2c8-430eec2a4472 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.587443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5dbafb8c-b10b-4845-b2c8-430eec2a4472 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.340717 May 25 16:22:41.591260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 397/812] 10.222.0.45 () {58 vars in 1274 bytes} [Wed May 25 16:22:41 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/os-interface => generated 285 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:22:41.591523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b17c2319-54e7-4f95-9eb8-ced903e35961 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:41.592782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-052ecddf-ad86-48c4-81c2-7fdfa513feee tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:41.593242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-052ecddf-ad86-48c4-81c2-7fdfa513feee tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:41.599472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b17c2319-54e7-4f95-9eb8-ced903e35961 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.425270 May 25 16:22:41.599636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-052ecddf-ad86-48c4-81c2-7fdfa513feee tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.609360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 416/813] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:41 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:41.609561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-052ecddf-ad86-48c4-81c2-7fdfa513feee tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.609561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-052ecddf-ad86-48c4-81c2-7fdfa513feee tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.609561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-052ecddf-ad86-48c4-81c2-7fdfa513feee tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.609788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b8f7f6cb-bf99-4683-8b02-490e0eaeca85 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:41.609788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b8f7f6cb-bf99-4683-8b02-490e0eaeca85 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:41.614134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b8f7f6cb-bf99-4683-8b02-490e0eaeca85 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.614732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b8f7f6cb-bf99-4683-8b02-490e0eaeca85 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.615536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b8f7f6cb-bf99-4683-8b02-490e0eaeca85 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.615965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b8f7f6cb-bf99-4683-8b02-490e0eaeca85 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.645790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b8f7f6cb-bf99-4683-8b02-490e0eaeca85 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044626 May 25 16:22:41.646612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 417/814] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:41 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:41.682699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:41.693535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-052ecddf-ad86-48c4-81c2-7fdfa513feee tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:41.701408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-052ecddf-ad86-48c4-81c2-7fdfa513feee tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.112378 May 25 16:22:41.701970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 398/815] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:41 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:41.718245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-47422785-9217-4a1f-a374-28c099a34cc2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:41.719188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-47422785-9217-4a1f-a374-28c099a34cc2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:41.726261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47422785-9217-4a1f-a374-28c099a34cc2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.726705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47422785-9217-4a1f-a374-28c099a34cc2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.727488 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47422785-9217-4a1f-a374-28c099a34cc2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:41.727919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47422785-9217-4a1f-a374-28c099a34cc2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:41.757225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-47422785-9217-4a1f-a374-28c099a34cc2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043465 May 25 16:22:41.758019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 418/816] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:42.597764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=8 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:42.617009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6cbf4c93-de9b-4d90-8b75-870ec018b7de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:42.617795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6cbf4c93-de9b-4d90-8b75-870ec018b7de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:42.625011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cbf4c93-de9b-4d90-8b75-870ec018b7de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:42.625522 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cbf4c93-de9b-4d90-8b75-870ec018b7de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:42.626341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cbf4c93-de9b-4d90-8b75-870ec018b7de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:42.626872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cbf4c93-de9b-4d90-8b75-870ec018b7de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:42.654560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-6970964e-b8a7-4c2b-bb10-2bc1a86c739a req-a328c278-830b-43a4-b049-4db85c25cea1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c98878d3-b9b6-4688-abea-b14c2150fb4a", "tag": "9beaea24-9678-4ef5-8aee-b08193e580ba"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:42.666542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-6970964e-b8a7-4c2b-bb10-2bc1a86c739a req-a328c278-830b-43a4-b049-4db85c25cea1 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:42.667083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-6970964e-b8a7-4c2b-bb10-2bc1a86c739a req-a328c278-830b-43a4-b049-4db85c25cea1 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:42.692103 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-6970964e-b8a7-4c2b-bb10-2bc1a86c739a req-a328c278-830b-43a4-b049-4db85c25cea1 service nova] Creating event network-changed:9beaea24-9678-4ef5-8aee-b08193e580ba for instance c98878d3-b9b6-4688-abea-b14c2150fb4a on hv-cinder-84327 May 25 16:22:42.697430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-6970964e-b8a7-4c2b-bb10-2bc1a86c739a req-a328c278-830b-43a4-b049-4db85c25cea1 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.104138 May 25 16:22:42.698132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 399/817] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:22:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:42.704246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c4394dd4-32be-4337-8e74-9ce77c470c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:42.704632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c4394dd4-32be-4337-8e74-9ce77c470c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:42.720410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4394dd4-32be-4337-8e74-9ce77c470c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:42.721132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4394dd4-32be-4337-8e74-9ce77c470c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:42.722372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4394dd4-32be-4337-8e74-9ce77c470c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:42.722845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4394dd4-32be-4337-8e74-9ce77c470c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:42.761619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c4394dd4-32be-4337-8e74-9ce77c470c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062187 May 25 16:22:42.763292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 400/818] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:42 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:42.766960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6cbf4c93-de9b-4d90-8b75-870ec018b7de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:42.772809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3e33d288-47d5-4a13-bec8-12380565f989 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:42.773928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3e33d288-47d5-4a13-bec8-12380565f989 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:42.776050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6cbf4c93-de9b-4d90-8b75-870ec018b7de tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.162891 May 25 16:22:42.780505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e33d288-47d5-4a13-bec8-12380565f989 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:42.780932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 419/819] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:42 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:22:42.781060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e33d288-47d5-4a13-bec8-12380565f989 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:42.781907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e33d288-47d5-4a13-bec8-12380565f989 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:42.782411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e33d288-47d5-4a13-bec8-12380565f989 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:42.789835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0021a6ab-bc4b-4952-be35-78c221826aa3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:42.792259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0021a6ab-bc4b-4952-be35-78c221826aa3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:42.799555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0021a6ab-bc4b-4952-be35-78c221826aa3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:42.800348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0021a6ab-bc4b-4952-be35-78c221826aa3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:42.801326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0021a6ab-bc4b-4952-be35-78c221826aa3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:42.802071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0021a6ab-bc4b-4952-be35-78c221826aa3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:42.849569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0021a6ab-bc4b-4952-be35-78c221826aa3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.066856 May 25 16:22:42.850693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 420/820] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:42.858091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3e33d288-47d5-4a13-bec8-12380565f989 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:42.866532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3e33d288-47d5-4a13-bec8-12380565f989 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.097955 May 25 16:22:42.867338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 401/821] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:42 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:43.775883 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-85d2473f-e483-4d55-89bc-7f34ef4dc480 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:43.777015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-85d2473f-e483-4d55-89bc-7f34ef4dc480 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:43.785649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85d2473f-e483-4d55-89bc-7f34ef4dc480 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:43.786217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85d2473f-e483-4d55-89bc-7f34ef4dc480 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:43.787199 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85d2473f-e483-4d55-89bc-7f34ef4dc480 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:43.787820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85d2473f-e483-4d55-89bc-7f34ef4dc480 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:43.794151 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d14b5af3-bca0-46f2-9994-f11c749d33f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:43.794871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d14b5af3-bca0-46f2-9994-f11c749d33f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:43.801739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d14b5af3-bca0-46f2-9994-f11c749d33f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:43.802208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d14b5af3-bca0-46f2-9994-f11c749d33f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:43.802975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d14b5af3-bca0-46f2-9994-f11c749d33f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:43.803448 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d14b5af3-bca0-46f2-9994-f11c749d33f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:43.806152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:43.819801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-85d2473f-e483-4d55-89bc-7f34ef4dc480 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048979 May 25 16:22:43.820366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 421/822] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:43 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:43.861593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7ee82f61-8b95-4818-a8f8-daf77fbe6c9a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:43.862543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7ee82f61-8b95-4818-a8f8-daf77fbe6c9a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:43.869077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ee82f61-8b95-4818-a8f8-daf77fbe6c9a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:43.869525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ee82f61-8b95-4818-a8f8-daf77fbe6c9a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:43.870297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ee82f61-8b95-4818-a8f8-daf77fbe6c9a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:43.870727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ee82f61-8b95-4818-a8f8-daf77fbe6c9a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:43.901764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7ee82f61-8b95-4818-a8f8-daf77fbe6c9a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044126 May 25 16:22:43.902347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 422/823] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 25 16:22:43.909280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f8d8d201-b299-4b2c-ab47-c9b2be1f1685 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:43.909887 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f8d8d201-b299-4b2c-ab47-c9b2be1f1685 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:43.916481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8d8d201-b299-4b2c-ab47-c9b2be1f1685 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:43.916937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8d8d201-b299-4b2c-ab47-c9b2be1f1685 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:43.917710 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8d8d201-b299-4b2c-ab47-c9b2be1f1685 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:43.918145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8d8d201-b299-4b2c-ab47-c9b2be1f1685 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:43.946095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d14b5af3-bca0-46f2-9994-f11c749d33f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:43.954167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d14b5af3-bca0-46f2-9994-f11c749d33f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.164337 May 25 16:22:43.954733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 402/824] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:43 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:43.981508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f8d8d201-b299-4b2c-ab47-c9b2be1f1685 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:43.989258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f8d8d201-b299-4b2c-ab47-c9b2be1f1685 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.085962 May 25 16:22:43.989822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 423/825] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:43 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:44.835471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9f8db558-e0ae-46eb-8111-c4c6447a6ccf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:44.836661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9f8db558-e0ae-46eb-8111-c4c6447a6ccf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:44.843974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f8db558-e0ae-46eb-8111-c4c6447a6ccf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:44.844424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f8db558-e0ae-46eb-8111-c4c6447a6ccf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:44.845192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f8db558-e0ae-46eb-8111-c4c6447a6ccf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:44.845653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f8db558-e0ae-46eb-8111-c4c6447a6ccf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:44.875781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9f8db558-e0ae-46eb-8111-c4c6447a6ccf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047295 May 25 16:22:44.876494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 403/826] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:44 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:44.913840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-50289313-a85e-4741-9f0f-f7820bf74757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:44.914788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-50289313-a85e-4741-9f0f-f7820bf74757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:44.922323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50289313-a85e-4741-9f0f-f7820bf74757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:44.922772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50289313-a85e-4741-9f0f-f7820bf74757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:44.923560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50289313-a85e-4741-9f0f-f7820bf74757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:44.923993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50289313-a85e-4741-9f0f-f7820bf74757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:44.953787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-50289313-a85e-4741-9f0f-f7820bf74757 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043997 May 25 16:22:44.954342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 424/827] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:44 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:44.974436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f20d3af3-955f-4281-ac99-09ebffea162c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:44.975494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f20d3af3-955f-4281-ac99-09ebffea162c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:44.984563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f20d3af3-955f-4281-ac99-09ebffea162c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:44.985230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f20d3af3-955f-4281-ac99-09ebffea162c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:44.986445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f20d3af3-955f-4281-ac99-09ebffea162c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:44.987091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f20d3af3-955f-4281-ac99-09ebffea162c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:45.005797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-272a3444-9cdf-4253-9916-dc6b957ed49f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:45.006440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-272a3444-9cdf-4253-9916-dc6b957ed49f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:45.012762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-272a3444-9cdf-4253-9916-dc6b957ed49f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:45.013211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-272a3444-9cdf-4253-9916-dc6b957ed49f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:45.013966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-272a3444-9cdf-4253-9916-dc6b957ed49f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:45.014375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-272a3444-9cdf-4253-9916-dc6b957ed49f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:45.086847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-272a3444-9cdf-4253-9916-dc6b957ed49f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:45.094531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-272a3444-9cdf-4253-9916-dc6b957ed49f tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.092527 May 25 16:22:45.095070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 425/828] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:45 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:45.143443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f20d3af3-955f-4281-ac99-09ebffea162c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:45.151785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f20d3af3-955f-4281-ac99-09ebffea162c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.183822 May 25 16:22:45.152320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 404/829] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:44 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:45.892847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-753cd59e-897f-4672-949d-396c55901f91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:45.894301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-753cd59e-897f-4672-949d-396c55901f91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:45.902319 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-753cd59e-897f-4672-949d-396c55901f91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:45.902769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-753cd59e-897f-4672-949d-396c55901f91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:45.903538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-753cd59e-897f-4672-949d-396c55901f91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:45.903992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-753cd59e-897f-4672-949d-396c55901f91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:45.935176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-753cd59e-897f-4672-949d-396c55901f91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049326 May 25 16:22:45.935917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 426/830] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:45 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:45.968672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b131216d-8bec-4209-bddc-6ac7e9ac45d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:45.970141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b131216d-8bec-4209-bddc-6ac7e9ac45d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:45.978775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b131216d-8bec-4209-bddc-6ac7e9ac45d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:45.979248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b131216d-8bec-4209-bddc-6ac7e9ac45d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:45.980009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b131216d-8bec-4209-bddc-6ac7e9ac45d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:45.980440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b131216d-8bec-4209-bddc-6ac7e9ac45d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:46.011078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b131216d-8bec-4209-bddc-6ac7e9ac45d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048984 May 25 16:22:46.011656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 405/831] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:45 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:46.110997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-94b8aa7f-07be-483e-814d-16da312071e6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:46.111644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-94b8aa7f-07be-483e-814d-16da312071e6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:46.118198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94b8aa7f-07be-483e-814d-16da312071e6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:46.118650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94b8aa7f-07be-483e-814d-16da312071e6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:46.119440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94b8aa7f-07be-483e-814d-16da312071e6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:46.119874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-94b8aa7f-07be-483e-814d-16da312071e6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:46.168768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b3676e19-21d2-42cd-a68d-ef8bc8230c1a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:46.169398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b3676e19-21d2-42cd-a68d-ef8bc8230c1a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:46.175745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b3676e19-21d2-42cd-a68d-ef8bc8230c1a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:46.176290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b3676e19-21d2-42cd-a68d-ef8bc8230c1a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:46.177120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b3676e19-21d2-42cd-a68d-ef8bc8230c1a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:46.177623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b3676e19-21d2-42cd-a68d-ef8bc8230c1a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:46.181791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-94b8aa7f-07be-483e-814d-16da312071e6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:46.189464 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-94b8aa7f-07be-483e-814d-16da312071e6 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.082398 May 25 16:22:46.190053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 427/832] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:46 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:46.312572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b3676e19-21d2-42cd-a68d-ef8bc8230c1a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:46.320958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b3676e19-21d2-42cd-a68d-ef8bc8230c1a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.156470 May 25 16:22:46.321501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 406/833] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:46 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:46.949197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2b810082-a4fc-4cab-8f15-95efa68648d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:46.950127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2b810082-a4fc-4cab-8f15-95efa68648d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:46.956940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2b810082-a4fc-4cab-8f15-95efa68648d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:46.957391 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2b810082-a4fc-4cab-8f15-95efa68648d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:46.958147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2b810082-a4fc-4cab-8f15-95efa68648d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:46.958662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2b810082-a4fc-4cab-8f15-95efa68648d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:46.978967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:46.994944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2b810082-a4fc-4cab-8f15-95efa68648d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049764 May 25 16:22:46.995542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 428/834] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:46 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:22:47.022406 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-046d460e-8a88-411d-9de0-810a5e7bdadd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:47.023365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-046d460e-8a88-411d-9de0-810a5e7bdadd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:47.030271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-046d460e-8a88-411d-9de0-810a5e7bdadd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:47.030701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-046d460e-8a88-411d-9de0-810a5e7bdadd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:47.031518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-046d460e-8a88-411d-9de0-810a5e7bdadd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:47.031950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-046d460e-8a88-411d-9de0-810a5e7bdadd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:47.061369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-046d460e-8a88-411d-9de0-810a5e7bdadd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043233 May 25 16:22:47.061934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 407/835] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:47.210375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0ddf2e64-4c75-4afb-822d-0c29c218f150 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:47.210927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0ddf2e64-4c75-4afb-822d-0c29c218f150 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:47.217837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ddf2e64-4c75-4afb-822d-0c29c218f150 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:47.218308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ddf2e64-4c75-4afb-822d-0c29c218f150 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:47.219173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ddf2e64-4c75-4afb-822d-0c29c218f150 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:47.219643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ddf2e64-4c75-4afb-822d-0c29c218f150 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:47.282321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ddf2e64-4c75-4afb-822d-0c29c218f150 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:47.290221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0ddf2e64-4c75-4afb-822d-0c29c218f150 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.086288 May 25 16:22:47.291102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 429/836] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:47 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:47.342116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e357829a-4872-4a23-b0a2-a93590383601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:47.343137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e357829a-4872-4a23-b0a2-a93590383601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:47.353400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e357829a-4872-4a23-b0a2-a93590383601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:47.354159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e357829a-4872-4a23-b0a2-a93590383601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:47.355568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e357829a-4872-4a23-b0a2-a93590383601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:47.356345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e357829a-4872-4a23-b0a2-a93590383601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:47.493992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e357829a-4872-4a23-b0a2-a93590383601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:47.502671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e357829a-4872-4a23-b0a2-a93590383601 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1716 microversion: 2.1 time: 0.166948 May 25 16:22:47.503260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 408/837] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:47 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1716 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:47.563616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:47.564250 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:47.571499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:47.571949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:47.572721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:47.573151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:48.007365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-75062057-aadb-486f-b48d-d09a7240972c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:48.008311 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-75062057-aadb-486f-b48d-d09a7240972c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:48.014808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75062057-aadb-486f-b48d-d09a7240972c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:48.015284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75062057-aadb-486f-b48d-d09a7240972c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:48.016107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75062057-aadb-486f-b48d-d09a7240972c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:48.016543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75062057-aadb-486f-b48d-d09a7240972c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:48.054504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-75062057-aadb-486f-b48d-d09a7240972c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051152 May 25 16:22:48.055215 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 409/838] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:48 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:48.073215 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-189f07d6-dea3-4e18-95c0-bf3bc47feda7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:48.074138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-189f07d6-dea3-4e18-95c0-bf3bc47feda7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:48.080828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-189f07d6-dea3-4e18-95c0-bf3bc47feda7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:48.081291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-189f07d6-dea3-4e18-95c0-bf3bc47feda7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:48.082057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-189f07d6-dea3-4e18-95c0-bf3bc47feda7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:48.082494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-189f07d6-dea3-4e18-95c0-bf3bc47feda7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:48.112858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-189f07d6-dea3-4e18-95c0-bf3bc47feda7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043510 May 25 16:22:48.113456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 410/839] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:48.311178 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4a893baa-e426-4033-a402-81e166ca0725 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:48.312219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4a893baa-e426-4033-a402-81e166ca0725 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:48.321276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a893baa-e426-4033-a402-81e166ca0725 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:48.321722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a893baa-e426-4033-a402-81e166ca0725 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:48.322477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a893baa-e426-4033-a402-81e166ca0725 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:48.322911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a893baa-e426-4033-a402-81e166ca0725 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:48.389955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4a893baa-e426-4033-a402-81e166ca0725 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:48.401531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4a893baa-e426-4033-a402-81e166ca0725 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.096481 May 25 16:22:48.402394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 411/840] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:48 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:49.071046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1695800d-4361-4939-842a-18dd64c76efb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:49.072037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1695800d-4361-4939-842a-18dd64c76efb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:49.078787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1695800d-4361-4939-842a-18dd64c76efb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:49.079309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1695800d-4361-4939-842a-18dd64c76efb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:49.080099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1695800d-4361-4939-842a-18dd64c76efb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:49.080560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1695800d-4361-4939-842a-18dd64c76efb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:49.111334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1695800d-4361-4939-842a-18dd64c76efb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045927 May 25 16:22:49.111969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 412/841] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:49 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:49.124548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-83de9972-b5ed-4458-b0df-55b71432d598 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:49.125455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-83de9972-b5ed-4458-b0df-55b71432d598 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:49.131610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83de9972-b5ed-4458-b0df-55b71432d598 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:49.132062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83de9972-b5ed-4458-b0df-55b71432d598 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:49.132812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83de9972-b5ed-4458-b0df-55b71432d598 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:49.133242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83de9972-b5ed-4458-b0df-55b71432d598 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:49.162423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-83de9972-b5ed-4458-b0df-55b71432d598 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041543 May 25 16:22:49.163003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 413/842] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:49.426409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b946eb4c-280e-43ea-a6c6-d6a81013f000 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:49.427301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b946eb4c-280e-43ea-a6c6-d6a81013f000 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:49.434672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b946eb4c-280e-43ea-a6c6-d6a81013f000 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:49.435142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b946eb4c-280e-43ea-a6c6-d6a81013f000 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:49.435967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b946eb4c-280e-43ea-a6c6-d6a81013f000 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:49.436416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b946eb4c-280e-43ea-a6c6-d6a81013f000 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:49.505766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b946eb4c-280e-43ea-a6c6-d6a81013f000 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:49.513594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b946eb4c-280e-43ea-a6c6-d6a81013f000 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.093747 May 25 16:22:49.514154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 414/843] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:49 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:50.124422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-54d3af00-1a5a-4f5e-af06-cf91eaeff2ec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:50.125399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-54d3af00-1a5a-4f5e-af06-cf91eaeff2ec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:50.132873 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-54d3af00-1a5a-4f5e-af06-cf91eaeff2ec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:50.133360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-54d3af00-1a5a-4f5e-af06-cf91eaeff2ec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:50.134183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-54d3af00-1a5a-4f5e-af06-cf91eaeff2ec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:50.134641 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-54d3af00-1a5a-4f5e-af06-cf91eaeff2ec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:50.164931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-54d3af00-1a5a-4f5e-af06-cf91eaeff2ec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045063 May 25 16:22:50.165805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 415/844] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:50 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:50.173757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1c563527-cc43-47ac-92f3-fe811f1fa487 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:50.174664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1c563527-cc43-47ac-92f3-fe811f1fa487 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:50.180819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c563527-cc43-47ac-92f3-fe811f1fa487 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:50.181279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c563527-cc43-47ac-92f3-fe811f1fa487 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:50.182029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c563527-cc43-47ac-92f3-fe811f1fa487 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:50.182471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c563527-cc43-47ac-92f3-fe811f1fa487 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:50.211591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1c563527-cc43-47ac-92f3-fe811f1fa487 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041523 May 25 16:22:50.212141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 416/845] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:50 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:50.534516 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-452d2dfa-725d-4b9b-ad23-e20526594fb7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:50.535580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-452d2dfa-725d-4b9b-ad23-e20526594fb7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:50.543198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-452d2dfa-725d-4b9b-ad23-e20526594fb7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:50.543686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-452d2dfa-725d-4b9b-ad23-e20526594fb7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:50.544450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-452d2dfa-725d-4b9b-ad23-e20526594fb7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:50.544896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-452d2dfa-725d-4b9b-ad23-e20526594fb7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:50.606663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-452d2dfa-725d-4b9b-ad23-e20526594fb7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:50.614490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-452d2dfa-725d-4b9b-ad23-e20526594fb7 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.086449 May 25 16:22:50.615071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 417/846] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:50 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:51.182146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0a1cff5d-68d1-4e43-a316-e0bbe32e7484 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:51.183747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0a1cff5d-68d1-4e43-a316-e0bbe32e7484 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:51.194420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0a1cff5d-68d1-4e43-a316-e0bbe32e7484 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:51.194856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0a1cff5d-68d1-4e43-a316-e0bbe32e7484 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:51.195657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0a1cff5d-68d1-4e43-a316-e0bbe32e7484 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:51.196093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0a1cff5d-68d1-4e43-a316-e0bbe32e7484 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:51.220888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:51.228464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0a1cff5d-68d1-4e43-a316-e0bbe32e7484 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053480 May 25 16:22:51.229063 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 418/847] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:51 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:51.233815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3b3ed009-4ee3-473d-a399-5e9b6a594902 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:51.234747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3b3ed009-4ee3-473d-a399-5e9b6a594902 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:51.240472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b3ed009-4ee3-473d-a399-5e9b6a594902 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:51.240930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b3ed009-4ee3-473d-a399-5e9b6a594902 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:51.241671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b3ed009-4ee3-473d-a399-5e9b6a594902 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:51.242103 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b3ed009-4ee3-473d-a399-5e9b6a594902 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:51.270873 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3b3ed009-4ee3-473d-a399-5e9b6a594902 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040815 May 25 16:22:51.271425 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 419/848] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:51.635354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5b76b843-7e19-4872-bd7e-dccf7fbbb1a4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:51.636362 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5b76b843-7e19-4872-bd7e-dccf7fbbb1a4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:51.646463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b76b843-7e19-4872-bd7e-dccf7fbbb1a4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:51.647479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b76b843-7e19-4872-bd7e-dccf7fbbb1a4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:51.648288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b76b843-7e19-4872-bd7e-dccf7fbbb1a4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:51.648725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b76b843-7e19-4872-bd7e-dccf7fbbb1a4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:51.710734 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5b76b843-7e19-4872-bd7e-dccf7fbbb1a4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:51.718778 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5b76b843-7e19-4872-bd7e-dccf7fbbb1a4 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.090019 May 25 16:22:51.719545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 420/849] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:51 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:52.244440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9f64b772-c960-4c19-8c40-20e49249946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:52.245812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9f64b772-c960-4c19-8c40-20e49249946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:52.256844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f64b772-c960-4c19-8c40-20e49249946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:52.257291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f64b772-c960-4c19-8c40-20e49249946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:52.258056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f64b772-c960-4c19-8c40-20e49249946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:52.258489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f64b772-c960-4c19-8c40-20e49249946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:52.287695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9f64b772-c960-4c19-8c40-20e49249946c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050002 May 25 16:22:52.288229 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 421/850] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:52 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:52.292968 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-eab0c306-2bd8-4460-90a8-9022aebc1d72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:52.293857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-eab0c306-2bd8-4460-90a8-9022aebc1d72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:52.299474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eab0c306-2bd8-4460-90a8-9022aebc1d72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:52.299924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eab0c306-2bd8-4460-90a8-9022aebc1d72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:52.300667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eab0c306-2bd8-4460-90a8-9022aebc1d72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:52.301099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eab0c306-2bd8-4460-90a8-9022aebc1d72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:52.330585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-eab0c306-2bd8-4460-90a8-9022aebc1d72 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041388 May 25 16:22:52.331131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 422/851] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:52.740022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-94b9a91d-c076-47a9-8ead-d5f4843a6194 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:52.741016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-94b9a91d-c076-47a9-8ead-d5f4843a6194 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:52.751410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94b9a91d-c076-47a9-8ead-d5f4843a6194 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:52.752168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94b9a91d-c076-47a9-8ead-d5f4843a6194 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:52.753423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94b9a91d-c076-47a9-8ead-d5f4843a6194 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:52.754173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94b9a91d-c076-47a9-8ead-d5f4843a6194 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:52.824843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-94b9a91d-c076-47a9-8ead-d5f4843a6194 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:52.832821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-94b9a91d-c076-47a9-8ead-d5f4843a6194 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.099534 May 25 16:22:52.833389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 423/852] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:52 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:53.303938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-32aa05da-f6a9-43e2-957b-cc6719f748ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:53.304887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-32aa05da-f6a9-43e2-957b-cc6719f748ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:53.311554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-32aa05da-f6a9-43e2-957b-cc6719f748ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:53.312003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-32aa05da-f6a9-43e2-957b-cc6719f748ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:53.312765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-32aa05da-f6a9-43e2-957b-cc6719f748ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:53.313192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-32aa05da-f6a9-43e2-957b-cc6719f748ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:53.342816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-32aa05da-f6a9-43e2-957b-cc6719f748ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045339 May 25 16:22:53.343293 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 424/853] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:53 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:53.347801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1a22c600-d301-41d7-94f4-89101d0f3f38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:53.348712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1a22c600-d301-41d7-94f4-89101d0f3f38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:53.354414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1a22c600-d301-41d7-94f4-89101d0f3f38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:53.354860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1a22c600-d301-41d7-94f4-89101d0f3f38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:53.355637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1a22c600-d301-41d7-94f4-89101d0f3f38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:53.356069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1a22c600-d301-41d7-94f4-89101d0f3f38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:53.384661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1a22c600-d301-41d7-94f4-89101d0f3f38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040390 May 25 16:22:53.385202 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 425/854] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:53 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:53.853848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6a48e42b-61f5-4be3-9d60-0fd8f37bf911 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:53.854861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6a48e42b-61f5-4be3-9d60-0fd8f37bf911 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:53.865160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a48e42b-61f5-4be3-9d60-0fd8f37bf911 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:53.866085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a48e42b-61f5-4be3-9d60-0fd8f37bf911 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:53.866825 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a48e42b-61f5-4be3-9d60-0fd8f37bf911 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:53.867297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a48e42b-61f5-4be3-9d60-0fd8f37bf911 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:53.931378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6a48e42b-61f5-4be3-9d60-0fd8f37bf911 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:53.939112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6a48e42b-61f5-4be3-9d60-0fd8f37bf911 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.091820 May 25 16:22:53.939714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 426/855] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:53 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:54.358380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1407d029-6626-4d69-bb52-a6323d14e8e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:54.360029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1407d029-6626-4d69-bb52-a6323d14e8e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:54.371184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1407d029-6626-4d69-bb52-a6323d14e8e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:54.371973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1407d029-6626-4d69-bb52-a6323d14e8e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:54.373237 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1407d029-6626-4d69-bb52-a6323d14e8e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:54.373964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1407d029-6626-4d69-bb52-a6323d14e8e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:54.417294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1407d029-6626-4d69-bb52-a6323d14e8e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065690 May 25 16:22:54.417850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 427/856] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:54 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:54.422496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bba3bc24-3fb1-4df6-ae00-07488fb18cfe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:54.423445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bba3bc24-3fb1-4df6-ae00-07488fb18cfe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:54.431516 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bba3bc24-3fb1-4df6-ae00-07488fb18cfe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:54.431967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bba3bc24-3fb1-4df6-ae00-07488fb18cfe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:54.432728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bba3bc24-3fb1-4df6-ae00-07488fb18cfe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:54.433161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bba3bc24-3fb1-4df6-ae00-07488fb18cfe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:54.462113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bba3bc24-3fb1-4df6-ae00-07488fb18cfe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043264 May 25 16:22:54.462653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 428/857] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:54.965107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-04866032-2d67-406a-89c0-0839b00fddab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:54.966122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-04866032-2d67-406a-89c0-0839b00fddab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:54.976065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04866032-2d67-406a-89c0-0839b00fddab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:54.976526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04866032-2d67-406a-89c0-0839b00fddab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:54.977291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04866032-2d67-406a-89c0-0839b00fddab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:54.977733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04866032-2d67-406a-89c0-0839b00fddab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:55.040934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-04866032-2d67-406a-89c0-0839b00fddab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:55.052144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-04866032-2d67-406a-89c0-0839b00fddab tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.093392 May 25 16:22:55.052874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 429/858] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:54 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:55.429335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f61590e5-a00e-4f70-87b0-4ed3cc1d7781 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:55.430479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f61590e5-a00e-4f70-87b0-4ed3cc1d7781 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:55.440706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f61590e5-a00e-4f70-87b0-4ed3cc1d7781 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:55.441338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f61590e5-a00e-4f70-87b0-4ed3cc1d7781 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:55.442354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f61590e5-a00e-4f70-87b0-4ed3cc1d7781 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:55.442896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f61590e5-a00e-4f70-87b0-4ed3cc1d7781 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:55.478245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f61590e5-a00e-4f70-87b0-4ed3cc1d7781 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053646 May 25 16:22:55.478805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 430/859] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:55 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:55.483556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cb42433d-9ce0-4b6e-a182-c814d36bf48c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:55.484469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cb42433d-9ce0-4b6e-a182-c814d36bf48c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:55.490185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb42433d-9ce0-4b6e-a182-c814d36bf48c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:55.490626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb42433d-9ce0-4b6e-a182-c814d36bf48c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:55.491548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb42433d-9ce0-4b6e-a182-c814d36bf48c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:55.491978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb42433d-9ce0-4b6e-a182-c814d36bf48c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:55.521264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cb42433d-9ce0-4b6e-a182-c814d36bf48c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041298 May 25 16:22:55.521808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 431/860] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:55 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:56.073488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6f8d405f-fdc0-478e-9fde-82718926079b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:56.075137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6f8d405f-fdc0-478e-9fde-82718926079b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:56.085247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f8d405f-fdc0-478e-9fde-82718926079b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:56.085715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f8d405f-fdc0-478e-9fde-82718926079b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:56.086461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f8d405f-fdc0-478e-9fde-82718926079b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:56.086898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f8d405f-fdc0-478e-9fde-82718926079b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:56.157526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6f8d405f-fdc0-478e-9fde-82718926079b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:56.166104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6f8d405f-fdc0-478e-9fde-82718926079b tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.099182 May 25 16:22:56.166670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 432/861] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:56 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:56.494359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-246248c0-bb5f-4d30-9829-81113e46c735 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:56.495915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-246248c0-bb5f-4d30-9829-81113e46c735 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:56.506947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-246248c0-bb5f-4d30-9829-81113e46c735 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:56.507435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-246248c0-bb5f-4d30-9829-81113e46c735 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:56.508201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-246248c0-bb5f-4d30-9829-81113e46c735 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:56.508635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-246248c0-bb5f-4d30-9829-81113e46c735 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:56.543449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-246248c0-bb5f-4d30-9829-81113e46c735 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055745 May 25 16:22:56.544203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 433/862] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:56 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:56.550298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-927da2bd-4a56-4763-960b-68a703c0234f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:56.551186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-927da2bd-4a56-4763-960b-68a703c0234f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:56.558534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-927da2bd-4a56-4763-960b-68a703c0234f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:56.558983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-927da2bd-4a56-4763-960b-68a703c0234f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:56.559766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-927da2bd-4a56-4763-960b-68a703c0234f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:56.560209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-927da2bd-4a56-4763-960b-68a703c0234f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:56.589132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-927da2bd-4a56-4763-960b-68a703c0234f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043628 May 25 16:22:56.589681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 434/863] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:56 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:57.189668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4d80c257-8921-4769-9c71-d6c7dbe70bd0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:57.190489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4d80c257-8921-4769-9c71-d6c7dbe70bd0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:57.199915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d80c257-8921-4769-9c71-d6c7dbe70bd0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:57.200524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d80c257-8921-4769-9c71-d6c7dbe70bd0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:57.201661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d80c257-8921-4769-9c71-d6c7dbe70bd0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:57.202261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d80c257-8921-4769-9c71-d6c7dbe70bd0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:57.225699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:57.272281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4d80c257-8921-4769-9c71-d6c7dbe70bd0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:57.280180 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4d80c257-8921-4769-9c71-d6c7dbe70bd0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.095531 May 25 16:22:57.280753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 435/864] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:57 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:57.555904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-607e32b6-3d7d-4c6f-a43b-f9a0e37e504c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:57.556878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-607e32b6-3d7d-4c6f-a43b-f9a0e37e504c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:57.563738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-607e32b6-3d7d-4c6f-a43b-f9a0e37e504c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:57.564190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-607e32b6-3d7d-4c6f-a43b-f9a0e37e504c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:57.564955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-607e32b6-3d7d-4c6f-a43b-f9a0e37e504c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:57.565388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-607e32b6-3d7d-4c6f-a43b-f9a0e37e504c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:57.568248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=35 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:57.591893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=13 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:57.596101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-607e32b6-3d7d-4c6f-a43b-f9a0e37e504c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044539 May 25 16:22:57.596665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 436/865] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:57 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:57.601389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-190bdcad-4a5d-4ca9-9df1-5369689ff0e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:57.602283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-190bdcad-4a5d-4ca9-9df1-5369689ff0e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:57.608409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-190bdcad-4a5d-4ca9-9df1-5369689ff0e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:57.608847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-190bdcad-4a5d-4ca9-9df1-5369689ff0e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:57.609616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-190bdcad-4a5d-4ca9-9df1-5369689ff0e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:57.610041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-190bdcad-4a5d-4ca9-9df1-5369689ff0e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:57.639087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-190bdcad-4a5d-4ca9-9df1-5369689ff0e0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041434 May 25 16:22:57.639662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 437/866] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:57.860399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0bcb532f-ec1a-4412-aa20-b1f078064dc5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1161984986"}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:57.861282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0bcb532f-ec1a-4412-aa20-b1f078064dc5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:57.869799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bcb532f-ec1a-4412-aa20-b1f078064dc5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:57.870270 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bcb532f-ec1a-4412-aa20-b1f078064dc5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:57.871019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bcb532f-ec1a-4412-aa20-b1f078064dc5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:57.871478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bcb532f-ec1a-4412-aa20-b1f078064dc5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:58.004653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.network.security_group_api [None req-0bcb532f-ec1a-4412-aa20-b1f078064dc5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Removing security group f5a3b2c8-8714-4d7d-b283-b8771f400e61 from port e2c0938e-f9fb-4796-87fa-103cbcfd2536 May 25 16:22:58.401123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0bcb532f-ec1a-4412-aa20-b1f078064dc5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "POST /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/action" status: 202 len: 0 microversion: 2.1 time: 0.548596 May 25 16:22:58.405528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 438/867] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:22:57 2022] POST /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/action => generated 0 bytes in 555 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:22:58.412342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-01c9a3e3-61b1-497b-80d4-c0c5cf9d77f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:58.413197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-01c9a3e3-61b1-497b-80d4-c0c5cf9d77f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:58.418578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:58.423518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01c9a3e3-61b1-497b-80d4-c0c5cf9d77f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:58.424279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01c9a3e3-61b1-497b-80d4-c0c5cf9d77f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:58.424751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01c9a3e3-61b1-497b-80d4-c0c5cf9d77f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:58.425187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01c9a3e3-61b1-497b-80d4-c0c5cf9d77f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:58.525016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-01c9a3e3-61b1-497b-80d4-c0c5cf9d77f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:58.534715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-01c9a3e3-61b1-497b-80d4-c0c5cf9d77f8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.127461 May 25 16:22:58.535518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 439/868] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:58 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:58.540910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-37937f89-6164-4bbf-879f-26abcb7f10a3 req-e78d09bf-e0f3-4e27-bc8b-5f3ba082b16c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d86354dc-f3d4-409d-b331-9f708947be2a", "tag": "e2c0938e-f9fb-4796-87fa-103cbcfd2536"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:22:58.550411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-37937f89-6164-4bbf-879f-26abcb7f10a3 req-e78d09bf-e0f3-4e27-bc8b-5f3ba082b16c service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:58.550991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-37937f89-6164-4bbf-879f-26abcb7f10a3 req-e78d09bf-e0f3-4e27-bc8b-5f3ba082b16c service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:58.563766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-37937f89-6164-4bbf-879f-26abcb7f10a3 req-e78d09bf-e0f3-4e27-bc8b-5f3ba082b16c service nova] Creating event network-changed:e2c0938e-f9fb-4796-87fa-103cbcfd2536 for instance d86354dc-f3d4-409d-b331-9f708947be2a on hv-cinder-84327 May 25 16:22:58.569535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-37937f89-6164-4bbf-879f-26abcb7f10a3 req-e78d09bf-e0f3-4e27-bc8b-5f3ba082b16c service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032759 May 25 16:22:58.570885 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 440/869] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:22:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:22:58.609241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5abd0d32-bf2e-4af9-bf89-3e9477eae105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:58.610156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5abd0d32-bf2e-4af9-bf89-3e9477eae105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:58.616874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5abd0d32-bf2e-4af9-bf89-3e9477eae105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:58.617547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5abd0d32-bf2e-4af9-bf89-3e9477eae105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:58.618572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5abd0d32-bf2e-4af9-bf89-3e9477eae105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:58.619256 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5abd0d32-bf2e-4af9-bf89-3e9477eae105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:58.665950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5abd0d32-bf2e-4af9-bf89-3e9477eae105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061487 May 25 16:22:58.666932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 441/870] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:22:58 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:22:58.673005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4220bbda-75b2-422e-8cf5-678e5dd4a9db tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:58.673189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4220bbda-75b2-422e-8cf5-678e5dd4a9db tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:58.679607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4220bbda-75b2-422e-8cf5-678e5dd4a9db tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:58.680013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4220bbda-75b2-422e-8cf5-678e5dd4a9db tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:58.680855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4220bbda-75b2-422e-8cf5-678e5dd4a9db tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:58.681292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4220bbda-75b2-422e-8cf5-678e5dd4a9db tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:58.714504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4220bbda-75b2-422e-8cf5-678e5dd4a9db tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046060 May 25 16:22:58.715263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 442/871] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:22:58 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:22:59.553557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-944ce41c-fec0-4f38-bcc3-2ca835d7bc4d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:59.554883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-944ce41c-fec0-4f38-bcc3-2ca835d7bc4d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:59.561748 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-944ce41c-fec0-4f38-bcc3-2ca835d7bc4d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:59.562299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-944ce41c-fec0-4f38-bcc3-2ca835d7bc4d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:59.563315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-944ce41c-fec0-4f38-bcc3-2ca835d7bc4d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:59.563847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-944ce41c-fec0-4f38-bcc3-2ca835d7bc4d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:59.634021 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-944ce41c-fec0-4f38-bcc3-2ca835d7bc4d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:22:59.642017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-944ce41c-fec0-4f38-bcc3-2ca835d7bc4d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.093389 May 25 16:22:59.642588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 443/872] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:22:59 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:22:59.650042 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c6e86623-52cd-40e0-9365-6d62b2fb93bf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:22:59.650536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c6e86623-52cd-40e0-9365-6d62b2fb93bf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:22:59.656554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6e86623-52cd-40e0-9365-6d62b2fb93bf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:59.657038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6e86623-52cd-40e0-9365-6d62b2fb93bf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:22:59.657806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6e86623-52cd-40e0-9365-6d62b2fb93bf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:22:59.658248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6e86623-52cd-40e0-9365-6d62b2fb93bf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:23:09.654240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=11 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:09.677122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=49 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:03.269524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:24:03.270583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:24:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:24:18.733436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:24:18.734047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:24:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 (ip 10.222.0.45) !!! May 25 16:24:26.100151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:24:26.101797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:24:26.103097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:24:26.103789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:24:26.105013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:24:26.107516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1102f207-f6d4-4bac-a7ab-bd046446e5db tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 98.546118 May 25 16:24:26.107516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 430/873] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:22:47 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 152 bytes in 98548 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:24:26.166716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-f8101744-0c94-4b71-9a08-08f8d6d3db8f req-6fb6d60b-5f27-4366-b4ec-a57cf0788d3b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d86354dc-f3d4-409d-b331-9f708947be2a", "tag": "e2c0938e-f9fb-4796-87fa-103cbcfd2536"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:24:26.174981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-f8101744-0c94-4b71-9a08-08f8d6d3db8f req-6fb6d60b-5f27-4366-b4ec-a57cf0788d3b service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.175453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-f8101744-0c94-4b71-9a08-08f8d6d3db8f req-6fb6d60b-5f27-4366-b4ec-a57cf0788d3b service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.186508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-f8101744-0c94-4b71-9a08-08f8d6d3db8f req-6fb6d60b-5f27-4366-b4ec-a57cf0788d3b service nova] Creating event network-changed:e2c0938e-f9fb-4796-87fa-103cbcfd2536 for instance d86354dc-f3d4-409d-b331-9f708947be2a on hv-cinder-84327 May 25 16:24:26.191760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-f8101744-0c94-4b71-9a08-08f8d6d3db8f req-6fb6d60b-5f27-4366-b4ec-a57cf0788d3b service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086859 May 25 16:24:26.192556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 431/874] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:24:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:24:26.197690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fc47b5f3-853f-4b4a-8cf5-d6ea0862c554 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:24:26.198706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fc47b5f3-853f-4b4a-8cf5-d6ea0862c554 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:24:26.206072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc47b5f3-853f-4b4a-8cf5-d6ea0862c554 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.207470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc47b5f3-853f-4b4a-8cf5-d6ea0862c554 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.208339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc47b5f3-853f-4b4a-8cf5-d6ea0862c554 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.209526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc47b5f3-853f-4b4a-8cf5-d6ea0862c554 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.244040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fc47b5f3-853f-4b4a-8cf5-d6ea0862c554 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050266 May 25 16:24:26.244924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 432/875] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:24:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:24:26.249382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dc155dac-ca17-44be-b058-e0a7ee102a51 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:24:26.250291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dc155dac-ca17-44be-b058-e0a7ee102a51 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:24:26.256740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc155dac-ca17-44be-b058-e0a7ee102a51 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.257310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc155dac-ca17-44be-b058-e0a7ee102a51 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.258088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc155dac-ca17-44be-b058-e0a7ee102a51 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.258540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc155dac-ca17-44be-b058-e0a7ee102a51 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.287966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dc155dac-ca17-44be-b058-e0a7ee102a51 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042320 May 25 16:24:26.288844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 433/876] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:24:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:24:26.319935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-c6e86623-52cd-40e0-9365-6d62b2fb93bf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:24:26.412845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aaf27ffb-acd7-4eb9-b76d-61aa68522243 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:24:26.413582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aaf27ffb-acd7-4eb9-b76d-61aa68522243 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:24:26.422399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aaf27ffb-acd7-4eb9-b76d-61aa68522243 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.422845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aaf27ffb-acd7-4eb9-b76d-61aa68522243 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.423847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aaf27ffb-acd7-4eb9-b76d-61aa68522243 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.424495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aaf27ffb-acd7-4eb9-b76d-61aa68522243 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.442394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c6e86623-52cd-40e0-9365-6d62b2fb93bf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 202 len: 0 microversion: 2.1 time: 86.795897 May 25 16:24:26.443249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 444/877] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:22:59 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 0 bytes in 86800 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:24:26.451332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-42c50a0e-c868-4d0f-9617-b56b99ce5067 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:24:26.452324 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-42c50a0e-c868-4d0f-9617-b56b99ce5067 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:24:26.459904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-42c50a0e-c868-4d0f-9617-b56b99ce5067 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.461912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-42c50a0e-c868-4d0f-9617-b56b99ce5067 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.462933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-42c50a0e-c868-4d0f-9617-b56b99ce5067 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.463521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-42c50a0e-c868-4d0f-9617-b56b99ce5067 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.499645 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-42c50a0e-c868-4d0f-9617-b56b99ce5067 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.054807 May 25 16:24:26.500620 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 445/878] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:24:26 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 146 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:24:26.507987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:24:26.508667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:24:26.516906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.517537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.518684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.519357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.527329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aaf27ffb-acd7-4eb9-b76d-61aa68522243 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:24:26.535807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aaf27ffb-acd7-4eb9-b76d-61aa68522243 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.245974 May 25 16:24:26.536609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 434/879] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:24:26 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:24:26.541761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:24:26.542487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:24:26.551049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.551690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:26.552470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:24:26.553262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:24:36.513427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=4 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:36.540562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=13,nova_cell1:INSERT=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:36.547692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=10 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:36.572700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=26 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:25:33.588093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:25:33.588741 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:25:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 (ip 10.222.0.45) !!! May 25 16:25:44.012453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:25:44.013329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:25:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:26:26.424871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:26:26.427025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:26:26.428553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:26:26.429452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:26:26.431129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:26:26.432527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b70def97-0523-4204-a160-fbf9ed06524d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 119.926348 May 25 16:26:26.432527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 446/880] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:24:26 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 152 bytes in 119928 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:26:26.434705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bae6d6cb-42bc-4125-ae4a-b6894f83a798 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:26:26.435691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bae6d6cb-42bc-4125-ae4a-b6894f83a798 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:26:26.442387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bae6d6cb-42bc-4125-ae4a-b6894f83a798 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.442830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bae6d6cb-42bc-4125-ae4a-b6894f83a798 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:26.443608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bae6d6cb-42bc-4125-ae4a-b6894f83a798 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.444021 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bae6d6cb-42bc-4125-ae4a-b6894f83a798 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:26.476284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:26:26.477346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bae6d6cb-42bc-4125-ae4a-b6894f83a798 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047009 May 25 16:26:26.477865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 447/881] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:26:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:26:26.483186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ea5eacc0-d59e-41be-b273-d6f6918040fc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:26:26.484320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ea5eacc0-d59e-41be-b273-d6f6918040fc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:26:26.492183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea5eacc0-d59e-41be-b273-d6f6918040fc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.492855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea5eacc0-d59e-41be-b273-d6f6918040fc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:26.494048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea5eacc0-d59e-41be-b273-d6f6918040fc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.495306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea5eacc0-d59e-41be-b273-d6f6918040fc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:26.530309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ea5eacc0-d59e-41be-b273-d6f6918040fc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051178 May 25 16:26:26.530876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 448/882] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:26:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:26:26.535364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0363b1d7-4699-4076-bdec-f6c5f7b565ef tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:26:26.535971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0363b1d7-4699-4076-bdec-f6c5f7b565ef tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:26:26.546673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0363b1d7-4699-4076-bdec-f6c5f7b565ef tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.547310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0363b1d7-4699-4076-bdec-f6c5f7b565ef tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:26.548087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0363b1d7-4699-4076-bdec-f6c5f7b565ef tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.548534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0363b1d7-4699-4076-bdec-f6c5f7b565ef tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:26.611779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0363b1d7-4699-4076-bdec-f6c5f7b565ef tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:26:26.619708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0363b1d7-4699-4076-bdec-f6c5f7b565ef tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.087842 May 25 16:26:26.620269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 449/883] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:26:26 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:26:26.624847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7c7f89af-3ba2-4b70-b4be-bbc37c8a6cc2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:26:26.625333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7c7f89af-3ba2-4b70-b4be-bbc37c8a6cc2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:26:26.630974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7c7f89af-3ba2-4b70-b4be-bbc37c8a6cc2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.631467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7c7f89af-3ba2-4b70-b4be-bbc37c8a6cc2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:26.632230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7c7f89af-3ba2-4b70-b4be-bbc37c8a6cc2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.632670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7c7f89af-3ba2-4b70-b4be-bbc37c8a6cc2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:26.656812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7c7f89af-3ba2-4b70-b4be-bbc37c8a6cc2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.035526 May 25 16:26:26.657431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 450/884] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:26:26 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 146 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:26:26.661814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:26:26.662300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:26:26.668430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.668717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:26.669498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:26:26.670047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:26:36.491037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=4 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:36.666337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=10 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:36.694467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=28 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:23.284004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:27:23.285031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:27:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 (ip 10.222.0.45) !!! May 25 16:27:31.026888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:27:31.027605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:27:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:27:38.744492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:27:38.745500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:27:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:27:43.402177 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:27:43.403030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:27:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 (ip 10.222.0.45) !!! May 25 16:28:26.660586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ae6239d2-2ef1-4ae6-8c0e-e5793a50b85c) May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:28:26.662888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:28:26.664365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ae6239d2-2ef1-4ae6-8c0e-e5793a50b85c) May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:28:26.665605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ae6239d2-2ef1-4ae6-8c0e-e5793a50b85c) May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:28:26.666718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ae6239d2-2ef1-4ae6-8c0e-e5793a50b85c) May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:28:26.668258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ae6239d2-2ef1-4ae6-8c0e-e5793a50b85c) May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:28:26.669233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ae6239d2-2ef1-4ae6-8c0e-e5793a50b85c) May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ae6239d2-2ef1-4ae6-8c0e-e5793a50b85c) May 25 16:28:26.670338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:28:26.674255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ae6239d2-2ef1-4ae6-8c0e-e5793a50b85c) {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:28:26.675115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3acc0996-d99c-445e-b23e-167b824418c5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 240.137442 May 25 16:28:26.676012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 435/885] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:24:26 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 269 bytes in 240138 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:28:26.681619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:28:26.682100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:28:26.689007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:26.689444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:26.690191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:26.690612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:26.781385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:28:26.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:28:26.784266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:28:26.784927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:28:26.786430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:28:26.787631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-04e1ee44-f4e5-4c12-ba40-704bfcdf663b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 120.126124 May 25 16:28:26.787631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 451/886] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:26:26 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 152 bytes in 120127 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:28:26.793815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-20b8fa5d-3710-499f-bbef-e0a93d954396 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:28:26.794738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-20b8fa5d-3710-499f-bbef-e0a93d954396 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:28:26.803274 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20b8fa5d-3710-499f-bbef-e0a93d954396 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:26.803725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20b8fa5d-3710-499f-bbef-e0a93d954396 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:26.804488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20b8fa5d-3710-499f-bbef-e0a93d954396 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:26.804950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20b8fa5d-3710-499f-bbef-e0a93d954396 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:26.834030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-20b8fa5d-3710-499f-bbef-e0a93d954396 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045714 May 25 16:28:26.834511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 452/887] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:28:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:28:26.904103 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8885cf2f-1dcf-4840-9e82-5d6d1cbbea1b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:28:26.905075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8885cf2f-1dcf-4840-9e82-5d6d1cbbea1b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:28:26.911416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8885cf2f-1dcf-4840-9e82-5d6d1cbbea1b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:26.911958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8885cf2f-1dcf-4840-9e82-5d6d1cbbea1b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:26.912721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8885cf2f-1dcf-4840-9e82-5d6d1cbbea1b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:26.913185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8885cf2f-1dcf-4840-9e82-5d6d1cbbea1b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:26.944628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8885cf2f-1dcf-4840-9e82-5d6d1cbbea1b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.109074 May 25 16:28:26.945189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 453/888] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:28:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:28:26.949553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3717a4ff-fa26-4489-aaa7-1f74ea5ed285 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:28:26.950148 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3717a4ff-fa26-4489-aaa7-1f74ea5ed285 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:28:26.956062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3717a4ff-fa26-4489-aaa7-1f74ea5ed285 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:26.956499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3717a4ff-fa26-4489-aaa7-1f74ea5ed285 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:26.957246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3717a4ff-fa26-4489-aaa7-1f74ea5ed285 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:26.957671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3717a4ff-fa26-4489-aaa7-1f74ea5ed285 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:27.028095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3717a4ff-fa26-4489-aaa7-1f74ea5ed285 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:28:27.038562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3717a4ff-fa26-4489-aaa7-1f74ea5ed285 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.092411 May 25 16:28:27.039144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 454/889] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:28:26 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:28:27.119311 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9e6ce104-e9de-4330-8701-a1317a63ef85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:28:27.119809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9e6ce104-e9de-4330-8701-a1317a63ef85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:28:27.126294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e6ce104-e9de-4330-8701-a1317a63ef85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:27.126750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e6ce104-e9de-4330-8701-a1317a63ef85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:27.127685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e6ce104-e9de-4330-8701-a1317a63ef85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:27.128128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e6ce104-e9de-4330-8701-a1317a63ef85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:27.152048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9e6ce104-e9de-4330-8701-a1317a63ef85 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.111900 May 25 16:28:27.152601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 455/890] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:28:27 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 146 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:28:27.157228 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:28:27.157854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:28:27.163630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:27.164078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:27.164834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:28:27.165553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:28:36.686784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:36.714695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=4 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:37.161896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=10 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:37.183854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=28 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:53.600559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:28:53.601616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:28:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 (ip 10.222.0.45) !!! May 25 16:29:04.025867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:29:04.026797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:29:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:30:26.857024 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:30:26.858559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:30:26.860041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:30:26.860939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:30:26.862464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:30:26.863743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e03cff68-5ec6-474f-adbb-e03958734013 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 119.706568 May 25 16:30:26.863743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 456/891] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:28:27 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 152 bytes in 119708 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:30:26.943242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:30:27.006142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:30:27.006656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:30:27.013363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:30:27.013814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:30:27.014584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:30:27.015022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:30:36.948152 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=4 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:30:37.010815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:30:37.034754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=6 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:30:43.294694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:30:43.295823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:30:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:30:51.037716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:30:51.038596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:30:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:30:58.763598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:30:58.764363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:30:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:31:03.416016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:31:03.416016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:31:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:32:13.613109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:32:13.613838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:32:24.038303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:32:24.038867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:32:26.931333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a4c950c6-bd2c-4ca0-80bb-4ecb887879bd) May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:26.933626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:26.935038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a4c950c6-bd2c-4ca0-80bb-4ecb887879bd) May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:26.935920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a4c950c6-bd2c-4ca0-80bb-4ecb887879bd) May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:26.936963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a4c950c6-bd2c-4ca0-80bb-4ecb887879bd) May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:26.938158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a4c950c6-bd2c-4ca0-80bb-4ecb887879bd) May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:26.939135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a4c950c6-bd2c-4ca0-80bb-4ecb887879bd) May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a4c950c6-bd2c-4ca0-80bb-4ecb887879bd) May 25 16:32:26.940238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:26.941294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a4c950c6-bd2c-4ca0-80bb-4ecb887879bd) {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:26.941294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 240.258250 May 25 16:32:26.941294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:32:26.941294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (10.222.0.45) May 25 16:32:26.941294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: CRITICAL nova [None req-7b1474bc-effe-4534-a02b-588a9c45f282 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Unhandled error: OSError: write error May 25 16:32:26.941294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova OSError: write error May 25 16:32:26.941294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova May 25 16:32:26.941802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 436/892] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:28:26 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 0 bytes in 240260 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:26.941802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-86652e65-13c1-46ef-af66-375d3efcf32c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:26.943246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-86652e65-13c1-46ef-af66-375d3efcf32c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:26.949560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86652e65-13c1-46ef-af66-375d3efcf32c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:26.949999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86652e65-13c1-46ef-af66-375d3efcf32c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:26.950753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86652e65-13c1-46ef-af66-375d3efcf32c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:26.951231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86652e65-13c1-46ef-af66-375d3efcf32c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:26.980954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:26.981503 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-86652e65-13c1-46ef-af66-375d3efcf32c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043744 May 25 16:32:26.981615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments (ip 10.222.0.45) !!! May 25 16:32:26.981615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments (10.222.0.45) May 25 16:32:26.982173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: CRITICAL nova [None req-86652e65-13c1-46ef-af66-375d3efcf32c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Unhandled error: OSError: write error May 25 16:32:26.982173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova OSError: write error May 25 16:32:26.982173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova May 25 16:32:26.982478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 437/893] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:32:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 0 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:26.986820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-78c09bbd-b78c-438f-87da-81ab33425c71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:26.987789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-78c09bbd-b78c-438f-87da-81ab33425c71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:26.993467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78c09bbd-b78c-438f-87da-81ab33425c71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:26.994661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78c09bbd-b78c-438f-87da-81ab33425c71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:26.995569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78c09bbd-b78c-438f-87da-81ab33425c71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:26.996224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78c09bbd-b78c-438f-87da-81ab33425c71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.032679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-78c09bbd-b78c-438f-87da-81ab33425c71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044422 May 25 16:32:27.032679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments (ip 10.222.0.45) !!! May 25 16:32:27.032679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments (10.222.0.45) May 25 16:32:27.032679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: CRITICAL nova [None req-78c09bbd-b78c-438f-87da-81ab33425c71 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Unhandled error: OSError: write error May 25 16:32:27.032679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova OSError: write error May 25 16:32:27.032679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova May 25 16:32:27.032679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 438/894] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:32:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 0 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.034249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b6b329b6-716a-487d-9697-b521cd696c73 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.034249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b6b329b6-716a-487d-9697-b521cd696c73 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.039766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b6b329b6-716a-487d-9697-b521cd696c73 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.040087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-652d1a36-411e-469a-b5bc-1d102e0fb910) May 25 16:32:27.040529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b6b329b6-716a-487d-9697-b521cd696c73 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.041283 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b6b329b6-716a-487d-9697-b521cd696c73 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.041812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b6b329b6-716a-487d-9697-b521cd696c73 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:27.044715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:27.046196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-652d1a36-411e-469a-b5bc-1d102e0fb910) May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:27.046904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-652d1a36-411e-469a-b5bc-1d102e0fb910) May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:27.048135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-652d1a36-411e-469a-b5bc-1d102e0fb910) May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.049657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-652d1a36-411e-469a-b5bc-1d102e0fb910) May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.050802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-652d1a36-411e-469a-b5bc-1d102e0fb910) May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-652d1a36-411e-469a-b5bc-1d102e0fb910) May 25 16:32:27.052061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.053295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-652d1a36-411e-469a-b5bc-1d102e0fb910) {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:27.053295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 120.184208 May 25 16:32:27.053295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:32:27.053295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (10.222.0.45) May 25 16:32:27.053295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: CRITICAL nova [None req-4634de72-8f4a-42c6-836d-07bf34e37abd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Unhandled error: OSError: write error May 25 16:32:27.053295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova OSError: write error May 25 16:32:27.053295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova May 25 16:32:27.054152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 457/895] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:30:26 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 0 bytes in 120186 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.054152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-eeb87a98-0c56-4649-83e4-5f0753313730 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.054152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-eeb87a98-0c56-4649-83e4-5f0753313730 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.060026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eeb87a98-0c56-4649-83e4-5f0753313730 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.060470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eeb87a98-0c56-4649-83e4-5f0753313730 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.061235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eeb87a98-0c56-4649-83e4-5f0753313730 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.061677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eeb87a98-0c56-4649-83e4-5f0753313730 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.086931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-eeb87a98-0c56-4649-83e4-5f0753313730 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.038481 May 25 16:32:27.087473 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips (ip 10.222.0.45) !!! May 25 16:32:27.087473 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips (10.222.0.45) May 25 16:32:27.088044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: CRITICAL nova [None req-eeb87a98-0c56-4649-83e4-5f0753313730 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Unhandled error: OSError: write error May 25 16:32:27.088044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova OSError: write error May 25 16:32:27.088044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova May 25 16:32:27.088303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 458/896] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 0 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.092879 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.093363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.099203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.099654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.100404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.100837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.104134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b6b329b6-716a-487d-9697-b521cd696c73 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:32:27.111828 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b6b329b6-716a-487d-9697-b521cd696c73 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.081934 May 25 16:32:27.112337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 (ip 10.222.0.45) !!! May 25 16:32:27.112337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 (10.222.0.45) May 25 16:32:27.112913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: CRITICAL nova [None req-b6b329b6-716a-487d-9697-b521cd696c73 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Unhandled error: OSError: write error May 25 16:32:27.112913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova OSError: write error May 25 16:32:27.112913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova May 25 16:32:27.113178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 439/897] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 0 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.117590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.118090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.124062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.124510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.125374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.125824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.191446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:27.193022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:27.194388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:27.195097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.196346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:27.197451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 0.105293 May 25 16:32:27.197451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 (ip 10.222.0.45) !!! May 25 16:32:27.197451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 (10.222.0.45) May 25 16:32:27.197451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: CRITICAL nova [None req-1603c506-c2d4-4dba-959b-9e5b2e827d85 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Unhandled error: OSError: write error May 25 16:32:27.197451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova OSError: write error May 25 16:32:27.197451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova May 25 16:32:27.197451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 459/898] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:27 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 0 bytes in 107 msecs (HTTP/1.1 409) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.200318 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0fbbb116-8703-4c13-95f8-d77d1799c08c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.201287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0fbbb116-8703-4c13-95f8-d77d1799c08c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.206961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0fbbb116-8703-4c13-95f8-d77d1799c08c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.207429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0fbbb116-8703-4c13-95f8-d77d1799c08c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.208200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0fbbb116-8703-4c13-95f8-d77d1799c08c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.208630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0fbbb116-8703-4c13-95f8-d77d1799c08c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.215754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:27.238894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0fbbb116-8703-4c13-95f8-d77d1799c08c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042350 May 25 16:32:27.239445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments (ip 10.222.0.45) !!! May 25 16:32:27.239445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments (10.222.0.45) May 25 16:32:27.240099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: CRITICAL nova [None req-0fbbb116-8703-4c13-95f8-d77d1799c08c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Unhandled error: OSError: write error May 25 16:32:27.240099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova OSError: write error May 25 16:32:27.240099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova May 25 16:32:27.240099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 460/899] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 0 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.244583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1f8e1cb5-81dc-4a83-9810-025e0a4e9df2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.245588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1f8e1cb5-81dc-4a83-9810-025e0a4e9df2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.251313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1f8e1cb5-81dc-4a83-9810-025e0a4e9df2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.251761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1f8e1cb5-81dc-4a83-9810-025e0a4e9df2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.252514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1f8e1cb5-81dc-4a83-9810-025e0a4e9df2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.252945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1f8e1cb5-81dc-4a83-9810-025e0a4e9df2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.272211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-47bfef6a-cd3e-46c4-96d0-f2c17a23dec9) May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:27.274435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:27.275859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-47bfef6a-cd3e-46c4-96d0-f2c17a23dec9) May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:27.276472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-47bfef6a-cd3e-46c4-96d0-f2c17a23dec9) May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:27.277465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-47bfef6a-cd3e-46c4-96d0-f2c17a23dec9) May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.278567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-47bfef6a-cd3e-46c4-96d0-f2c17a23dec9) May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.279598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-47bfef6a-cd3e-46c4-96d0-f2c17a23dec9) May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-47bfef6a-cd3e-46c4-96d0-f2c17a23dec9) May 25 16:32:27.280636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.281615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-47bfef6a-cd3e-46c4-96d0-f2c17a23dec9) {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:27.281615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 0.161921 May 25 16:32:27.281615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (ip 10.222.0.45) !!! May 25 16:32:27.281615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 (10.222.0.45) May 25 16:32:27.281615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: CRITICAL nova [None req-292a7d7b-76e9-44b1-8bc8-abf31590f877 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Unhandled error: OSError: write error May 25 16:32:27.281615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova OSError: write error May 25 16:32:27.281615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova May 25 16:32:27.282076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 440/900] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:27 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 0 bytes in 163 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.282154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1f8e1cb5-81dc-4a83-9810-025e0a4e9df2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040282 May 25 16:32:27.282247 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8ff2e039-77f9-446e-a116-43e32899870e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.282332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments (ip 10.222.0.45) !!! May 25 16:32:27.282332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments (10.222.0.45) May 25 16:32:27.282467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8ff2e039-77f9-446e-a116-43e32899870e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.282551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: CRITICAL nova [None req-1f8e1cb5-81dc-4a83-9810-025e0a4e9df2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Unhandled error: OSError: write error May 25 16:32:27.282551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova OSError: write error May 25 16:32:27.282551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova May 25 16:32:27.282551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 461/901] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 0 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.286827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4a2a237a-999d-489f-adf3-28d0b11d3967 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.287344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4a2a237a-999d-489f-adf3-28d0b11d3967 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.288436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ff2e039-77f9-446e-a116-43e32899870e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.288886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ff2e039-77f9-446e-a116-43e32899870e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.289681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ff2e039-77f9-446e-a116-43e32899870e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.290123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ff2e039-77f9-446e-a116-43e32899870e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.293315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a2a237a-999d-489f-adf3-28d0b11d3967 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.293799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a2a237a-999d-489f-adf3-28d0b11d3967 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.294551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a2a237a-999d-489f-adf3-28d0b11d3967 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.294985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a2a237a-999d-489f-adf3-28d0b11d3967 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.317710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4a2a237a-999d-489f-adf3-28d0b11d3967 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.034196 May 25 16:32:27.318232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips (ip 10.222.0.45) !!! May 25 16:32:27.318232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips (10.222.0.45) May 25 16:32:27.318802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: CRITICAL nova [None req-4a2a237a-999d-489f-adf3-28d0b11d3967 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Unhandled error: OSError: write error May 25 16:32:27.318802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova OSError: write error May 25 16:32:27.318802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova May 25 16:32:27.319117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 462/902] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 0 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.323581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.324056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.329953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.330394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.331173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.331606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.360369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8ff2e039-77f9-446e-a116-43e32899870e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:32:27.370131 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8ff2e039-77f9-446e-a116-43e32899870e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1225 microversion: 2.1 time: 0.092030 May 25 16:32:27.370875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 (ip 10.222.0.45) !!! May 25 16:32:27.371087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:32:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 (10.222.0.45) May 25 16:32:27.371835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: CRITICAL nova [None req-8ff2e039-77f9-446e-a116-43e32899870e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Unhandled error: OSError: write error May 25 16:32:27.371835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova OSError: write error May 25 16:32:27.371835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova May 25 16:32:27.372079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 441/903] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 0 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 25 16:32:27.377130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.377698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.383959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.384530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.385636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.386243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.433689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:27.435925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:27.437234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:27.437877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.438997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.441288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:27.442220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9ca94e34-a8e0-45c1-9bd2-9d831d5d828c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 0.122381 May 25 16:32:27.443090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 463/904] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:27 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 152 bytes in 124 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:32:27.449591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3ba0b6b5-1f41-4efc-ab68-b496134c6995 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.450554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3ba0b6b5-1f41-4efc-ab68-b496134c6995 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.460923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ba0b6b5-1f41-4efc-ab68-b496134c6995 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.461370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ba0b6b5-1f41-4efc-ab68-b496134c6995 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.462168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ba0b6b5-1f41-4efc-ab68-b496134c6995 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.462597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ba0b6b5-1f41-4efc-ab68-b496134c6995 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.481232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:27.498616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3ba0b6b5-1f41-4efc-ab68-b496134c6995 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054170 May 25 16:32:27.499213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 464/905] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:27.504160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-185950a7-725a-45bc-8b9c-595e424ff7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.505254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-185950a7-725a-45bc-8b9c-595e424ff7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.513225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-185950a7-725a-45bc-8b9c-595e424ff7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.513621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-185950a7-725a-45bc-8b9c-595e424ff7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.514527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-185950a7-725a-45bc-8b9c-595e424ff7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.514961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-185950a7-725a-45bc-8b9c-595e424ff7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.540437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-38b803f8-483c-4f64-b845-eb348fa99bff) May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:27.544009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:27.545634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-38b803f8-483c-4f64-b845-eb348fa99bff) May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:27.546261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-38b803f8-483c-4f64-b845-eb348fa99bff) May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:27.547506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-38b803f8-483c-4f64-b845-eb348fa99bff) May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.548631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-38b803f8-483c-4f64-b845-eb348fa99bff) May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.549644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-38b803f8-483c-4f64-b845-eb348fa99bff) May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-38b803f8-483c-4f64-b845-eb348fa99bff) May 25 16:32:27.550651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:27.551694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-38b803f8-483c-4f64-b845-eb348fa99bff) {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:27.551792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-185950a7-725a-45bc-8b9c-595e424ff7e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044672 May 25 16:32:27.551879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7a1788b6-3308-44a5-8279-01539c9f54f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 0.172537 May 25 16:32:27.551962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 465/906] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:32:27.552043 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 442/907] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:27 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 269 bytes in 174 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:27.552122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.552122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.552280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-90c3858b-3589-44bf-a005-8b9afc6f44f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.552280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-90c3858b-3589-44bf-a005-8b9afc6f44f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.556613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.557061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.557360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90c3858b-3589-44bf-a005-8b9afc6f44f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.557823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90c3858b-3589-44bf-a005-8b9afc6f44f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.557928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.558394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.558585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90c3858b-3589-44bf-a005-8b9afc6f44f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.559022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90c3858b-3589-44bf-a005-8b9afc6f44f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.583416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-90c3858b-3589-44bf-a005-8b9afc6f44f2 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.036162 May 25 16:32:27.584213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 443/908] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 146 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:27.589072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.589547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.590013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lazy-loading 'fault' on Instance uuid b3b920a7-fa28-4150-b889-d9a2ea63bc16 {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:27.630505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7da83fc4029c4cdebca07c7247669bd0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01367a52e9104f85baec4b18ef8bf258', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:32:27.638116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-84b5e77e-2a20-4ac3-8c05-3d2abfbdbe0e tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 200 len: 1230 microversion: 2.1 time: 0.091486 May 25 16:32:27.638688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 466/909] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 1230 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:32:27.643298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.643795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.649501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.650007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.650716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.651264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.722052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fdf95c22-c7f6-41dc-ae2b-406ea1677956 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.723313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fdf95c22-c7f6-41dc-ae2b-406ea1677956 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.733794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fdf95c22-c7f6-41dc-ae2b-406ea1677956 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.734983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fdf95c22-c7f6-41dc-ae2b-406ea1677956 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.735959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fdf95c22-c7f6-41dc-ae2b-406ea1677956 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.736932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fdf95c22-c7f6-41dc-ae2b-406ea1677956 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.738419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:27.740084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:27.741651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:27.742458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:27.743790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.746784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:27.748366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8b7cd951-463a-4838-a5a4-075e87b679e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 0.108496 May 25 16:32:27.749010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 467/910] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:27 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 152 bytes in 110 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:32:27.756043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.756618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.764402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.764863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.765635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.766938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.826534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-fdf95c22-c7f6-41dc-ae2b-406ea1677956 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lazy-loading 'services' on Instance uuid 3fc5dcaa-3275-47eb-88ec-62b23505c91b {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:27.876461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:27.922660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fdf95c22-c7f6-41dc-ae2b-406ea1677956 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861" status: 202 len: 0 microversion: 2.1 time: 0.337096 May 25 16:32:27.923668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 444/911] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:27 2022] DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861 => generated 0 bytes in 339 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:32:27.930029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-78f90912-9ae8-48a3-884f-bb1844158e4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.931250 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-78f90912-9ae8-48a3-884f-bb1844158e4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.938385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78f90912-9ae8-48a3-884f-bb1844158e4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.938975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78f90912-9ae8-48a3-884f-bb1844158e4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.939738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78f90912-9ae8-48a3-884f-bb1844158e4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.939738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78f90912-9ae8-48a3-884f-bb1844158e4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.942174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5a077bf1-fcbb-4635-819f-25afd5f86836) May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:27.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:27.945953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5a077bf1-fcbb-4635-819f-25afd5f86836) May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:27.946589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5a077bf1-fcbb-4635-819f-25afd5f86836) May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:27.947690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5a077bf1-fcbb-4635-819f-25afd5f86836) May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.948992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5a077bf1-fcbb-4635-819f-25afd5f86836) May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.949992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5a077bf1-fcbb-4635-819f-25afd5f86836) May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5a077bf1-fcbb-4635-819f-25afd5f86836) May 25 16:32:27.951122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:27.952502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5a077bf1-fcbb-4635-819f-25afd5f86836) {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:27.952502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ec547928-a4f8-4ff5-8d0c-68ad239de1fe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 0.195372 May 25 16:32:27.952502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 468/912] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:27 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 269 bytes in 197 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:27.952502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fcec901e-a469-4aa5-87d7-8f12f13fe067 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.952869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fcec901e-a469-4aa5-87d7-8f12f13fe067 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.959402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fcec901e-a469-4aa5-87d7-8f12f13fe067 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.959856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fcec901e-a469-4aa5-87d7-8f12f13fe067 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.960596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fcec901e-a469-4aa5-87d7-8f12f13fe067 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.961259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fcec901e-a469-4aa5-87d7-8f12f13fe067 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.979306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-78f90912-9ae8-48a3-884f-bb1844158e4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054284 May 25 16:32:27.979876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 445/913] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:27.984497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:27.985187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:27.993851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fcec901e-a469-4aa5-87d7-8f12f13fe067 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046083 May 25 16:32:27.994707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 469/914] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:32:27.996477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.996948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:27.997723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:27.998152 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.001374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-75945467-20d0-4e5e-8c92-bb47f4542b61 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.001976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-75945467-20d0-4e5e-8c92-bb47f4542b61 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.008076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75945467-20d0-4e5e-8c92-bb47f4542b61 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.008640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75945467-20d0-4e5e-8c92-bb47f4542b61 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.009654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75945467-20d0-4e5e-8c92-bb47f4542b61 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.010191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75945467-20d0-4e5e-8c92-bb47f4542b61 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.012185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] HTTP exception thrown: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Request method failure captured: May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 HTTP/1.1 May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbWFeoxbZ3XsqIpF0DsfOI5naE8sdWXPvt2-XBM059bd-3IcPRs2zKgCatH24oxhBCqAGB9S8-2TnjMb2HOQB8fgsaN7fMFsDRgpSeLztlAjjNXbpOLdf3DjKzT0d5VX9i7usk6EiBa_tJtHYRJkUINtSZbIduMdV6TxGU5Onxayw4 May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:28.013881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 7da83fc4029c4cdebca07c7247669bd0 May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.015585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'b3b920a7-fa28-4150-b889-d9a2ea63bc16'} May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.018159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.019568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.020855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Returning 404 to user: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.020855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3db60b1a-d5da-498a-9961-e5988c2ed6b8 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 404 len: 111 microversion: 2.1 time: 0.034383 May 25 16:32:28.020855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 446/915] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:32:28.021457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.021950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.028172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.028622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.029381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.029844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.041553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-75945467-20d0-4e5e-8c92-bb47f4542b61 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.045704 May 25 16:32:28.042271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 470/916] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:32:27 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 146 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:28.046863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.047445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.053293 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.053785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.054612 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.055114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.111465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lazy-loading 'services' on Instance uuid 3fc5dcaa-3275-47eb-88ec-62b23505c91b {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:28.153655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:28.155091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:28.156456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.157223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.158493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.159477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e723cecf-250b-40b1-bda9-c902c1cc665c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 0.113623 May 25 16:32:28.159477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 471/917] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 152 bytes in 115 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:32:28.162592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.163077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.169478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.169927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.170685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.171122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.183897 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f68c0882-8ca1-4d88-af0c-93d15a0b749e) May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Request method failure captured: May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861 HTTP/1.1 May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlcTScjWaWhWZ_Y2deTS_iB2n7fhEPm9xpcRCdPE-r6ZMxY0ZuDpSToYfeI_E_5xfVHNIs52uYE6eqLoLyVrCNCdQhvtSm-zE7RKP38WrqQwdjdKcvzjavXk0OYwhbdCOQ2Ef36er8czPOW__xeJxnmKi5Ep7buFXy2Oju0Jbg9THQA May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:28.186366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_073368e447af465abb41aa556ee46dc9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/073368e447af465abb41aa556ee46dc9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: b53b45a485b14e0b9273c019dc9ec206 May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:32:28.187719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f68c0882-8ca1-4d88-af0c-93d15a0b749e) May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': '3fc5dcaa-3275-47eb-88ec-62b23505c91b', 'id': 'ca477fe6-273b-4b1f-a801-7dc44a982861'} May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.188308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f68c0882-8ca1-4d88-af0c-93d15a0b749e) May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.189292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f68c0882-8ca1-4d88-af0c-93d15a0b749e) May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.190363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f68c0882-8ca1-4d88-af0c-93d15a0b749e) May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.191387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f68c0882-8ca1-4d88-af0c-93d15a0b749e) May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f68c0882-8ca1-4d88-af0c-93d15a0b749e) May 25 16:32:28.192401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.193567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f68c0882-8ca1-4d88-af0c-93d15a0b749e) {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.193567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f8ea5699-9b6c-4870-99bb-fa36d1ac0fab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861" status: 400 len: 269 microversion: 2.1 time: 0.170642 May 25 16:32:28.193567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 447/918] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861 => generated 269 bytes in 172 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:28.193567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4f717287-9662-44aa-bb92-425f5fd1209e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.194062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4f717287-9662-44aa-bb92-425f5fd1209e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.199876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f717287-9662-44aa-bb92-425f5fd1209e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.200329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f717287-9662-44aa-bb92-425f5fd1209e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.201091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f717287-9662-44aa-bb92-425f5fd1209e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.201542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f717287-9662-44aa-bb92-425f5fd1209e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.232631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4f717287-9662-44aa-bb92-425f5fd1209e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043416 May 25 16:32:28.233171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 448/919] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:28.237975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ffb20838-198c-44b3-bfc2-66b63ee54a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.238875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ffb20838-198c-44b3-bfc2-66b63ee54a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.244527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ffb20838-198c-44b3-bfc2-66b63ee54a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.245077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ffb20838-198c-44b3-bfc2-66b63ee54a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.245839 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ffb20838-198c-44b3-bfc2-66b63ee54a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.246275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ffb20838-198c-44b3-bfc2-66b63ee54a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.254079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:28.274935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ffb20838-198c-44b3-bfc2-66b63ee54a53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040542 May 25 16:32:28.275580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 449/920] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:32:28.279936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.280543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.286150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.286606 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.287404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.287841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.301449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] HTTP exception thrown: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Request method failure captured: May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 HTTP/1.1 May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbWFeoxbZ3XsqIpF0DsfOI5naE8sdWXPvt2-XBM059bd-3IcPRs2zKgCatH24oxhBCqAGB9S8-2TnjMb2HOQB8fgsaN7fMFsDRgpSeLztlAjjNXbpOLdf3DjKzT0d5VX9i7usk6EiBa_tJtHYRJkUINtSZbIduMdV6TxGU5Onxayw4 May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:28.302979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 7da83fc4029c4cdebca07c7247669bd0 May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.304357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'b3b920a7-fa28-4150-b889-d9a2ea63bc16'} May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.305051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.306381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.307549 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Returning 404 to user: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.307549 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-120967d0-62cb-47f5-90f4-4302b24c5e66 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 404 len: 111 microversion: 2.1 time: 0.027864 May 25 16:32:28.307549 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 450/921] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:32:28.309337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-83a12b22-2819-419c-a59f-c4df2b353a52 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.309833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-83a12b22-2819-419c-a59f-c4df2b353a52 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.310515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-984e5345-95f7-4971-829d-f1276181966a) May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:28.312716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:28.314077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-984e5345-95f7-4971-829d-f1276181966a) May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.314696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-984e5345-95f7-4971-829d-f1276181966a) May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.315781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-984e5345-95f7-4971-829d-f1276181966a) May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.316867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-984e5345-95f7-4971-829d-f1276181966a) May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.317854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-984e5345-95f7-4971-829d-f1276181966a) May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-984e5345-95f7-4971-829d-f1276181966a) May 25 16:32:28.318938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.322428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-984e5345-95f7-4971-829d-f1276181966a) {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.322428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2cdaafac-ea33-4689-a2b9-5a4de7b166db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 0.155258 May 25 16:32:28.322428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 472/922] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 269 bytes in 156 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:28.322684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83a12b22-2819-419c-a59f-c4df2b353a52 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.322684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83a12b22-2819-419c-a59f-c4df2b353a52 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.322684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83a12b22-2819-419c-a59f-c4df2b353a52 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.322684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83a12b22-2819-419c-a59f-c4df2b353a52 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.323075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.323075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.325962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.326411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.327194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.327623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.340620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-83a12b22-2819-419c-a59f-c4df2b353a52 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.034741 May 25 16:32:28.341190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 451/923] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 146 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:28.345674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.346166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.351891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.352342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.353105 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.353538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.405224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:32:28.409705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lazy-loading 'services' on Instance uuid 3fc5dcaa-3275-47eb-88ec-62b23505c91b {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:28.451051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:28.452627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:28.454144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.455045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.456508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.457762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c285a0b7-4615-4d25-9ff6-c700cfa4ea0d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 0.111905 May 25 16:32:28.457762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 452/924] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 152 bytes in 113 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:32:28.459483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.459964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.466749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.467221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.468208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.468474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.480950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e8d90c51-b9c9-4e03-91d7-c21e65633250) May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Request method failure captured: May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861 HTTP/1.1 May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlcTScjWaWhWZ_Y2deTS_iB2n7fhEPm9xpcRCdPE-r6ZMxY0ZuDpSToYfeI_E_5xfVHNIs52uYE6eqLoLyVrCNCdQhvtSm-zE7RKP38WrqQwdjdKcvzjavXk0OYwhbdCOQ2Ef36er8czPOW__xeJxnmKi5Ep7buFXy2Oju0Jbg9THQA May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:28.483340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_073368e447af465abb41aa556ee46dc9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/073368e447af465abb41aa556ee46dc9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: b53b45a485b14e0b9273c019dc9ec206 May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:32:28.484685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e8d90c51-b9c9-4e03-91d7-c21e65633250) May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '3fc5dcaa-3275-47eb-88ec-62b23505c91b', 'id': 'ca477fe6-273b-4b1f-a801-7dc44a982861'} May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.485273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e8d90c51-b9c9-4e03-91d7-c21e65633250) May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.486599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e8d90c51-b9c9-4e03-91d7-c21e65633250) May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.488685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e8d90c51-b9c9-4e03-91d7-c21e65633250) May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.490725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e8d90c51-b9c9-4e03-91d7-c21e65633250) May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e8d90c51-b9c9-4e03-91d7-c21e65633250) May 25 16:32:28.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.496004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e8d90c51-b9c9-4e03-91d7-c21e65633250) {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.496004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8196461b-60ca-4a16-9cd7-7eea8f273b9b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861" status: 400 len: 269 microversion: 2.1 time: 0.169409 May 25 16:32:28.496004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 473/925] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861 => generated 269 bytes in 170 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:28.496004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ec35e187-19a1-4693-99e6-a79ca20c05e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.496646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ec35e187-19a1-4693-99e6-a79ca20c05e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.500989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec35e187-19a1-4693-99e6-a79ca20c05e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.500989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec35e187-19a1-4693-99e6-a79ca20c05e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.501252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec35e187-19a1-4693-99e6-a79ca20c05e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.501525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec35e187-19a1-4693-99e6-a79ca20c05e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.533292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ec35e187-19a1-4693-99e6-a79ca20c05e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046535 May 25 16:32:28.533872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 474/926] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:28.538462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-78d06527-e367-4a1a-b03f-fb93f7b945e7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.539406 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-78d06527-e367-4a1a-b03f-fb93f7b945e7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.545943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-78d06527-e367-4a1a-b03f-fb93f7b945e7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.546395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-78d06527-e367-4a1a-b03f-fb93f7b945e7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.547206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-78d06527-e367-4a1a-b03f-fb93f7b945e7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.547631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-78d06527-e367-4a1a-b03f-fb93f7b945e7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.554468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:32:28.560239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:28.577503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-78d06527-e367-4a1a-b03f-fb93f7b945e7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042546 May 25 16:32:28.578074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 475/927] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:32:28.582502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.583107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.589488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.589929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.590690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.591126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.605348 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] HTTP exception thrown: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Request method failure captured: May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 HTTP/1.1 May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlbWFeoxbZ3XsqIpF0DsfOI5naE8sdWXPvt2-XBM059bd-3IcPRs2zKgCatH24oxhBCqAGB9S8-2TnjMb2HOQB8fgsaN7fMFsDRgpSeLztlAjjNXbpOLdf3DjKzT0d5VX9i7usk6EiBa_tJtHYRJkUINtSZbIduMdV6TxGU5Onxayw4 May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:28.606957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 7da83fc4029c4cdebca07c7247669bd0 May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.608260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'b3b920a7-fa28-4150-b889-d9a2ea63bc16'} May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.608954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.610911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.612004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Returning 404 to user: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.612004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-451565b9-fc52-4926-beec-b2b3a2fd3ed1 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 404 len: 111 microversion: 2.1 time: 0.029377 May 25 16:32:28.612004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 476/928] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:32:28.613347 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-caa4e4ce-0f59-48e1-9ee0-d9d8ee9b3ef5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.613807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-caa4e4ce-0f59-48e1-9ee0-d9d8ee9b3ef5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.619659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-caa4e4ce-0f59-48e1-9ee0-d9d8ee9b3ef5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.620159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-caa4e4ce-0f59-48e1-9ee0-d9d8ee9b3ef5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.620889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-caa4e4ce-0f59-48e1-9ee0-d9d8ee9b3ef5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.621365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-caa4e4ce-0f59-48e1-9ee0-d9d8ee9b3ef5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.632126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-46ee66e8-535c-4a07-ae55-17479662c3a0) May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:28.634355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:28.635782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-46ee66e8-535c-4a07-ae55-17479662c3a0) May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.636416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-46ee66e8-535c-4a07-ae55-17479662c3a0) May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.637445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-46ee66e8-535c-4a07-ae55-17479662c3a0) May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.638532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-46ee66e8-535c-4a07-ae55-17479662c3a0) May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.639544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-46ee66e8-535c-4a07-ae55-17479662c3a0) May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-46ee66e8-535c-4a07-ae55-17479662c3a0) May 25 16:32:28.640625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.641685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-46ee66e8-535c-4a07-ae55-17479662c3a0) {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.641685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-11010f97-b019-46cf-a67c-970b0f5b1c5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 0.180094 May 25 16:32:28.641685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 453/929] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 269 bytes in 181 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:28.641685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.641985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.644843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-caa4e4ce-0f59-48e1-9ee0-d9d8ee9b3ef5 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.035005 May 25 16:32:28.645407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 477/930] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 146 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:28.648287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.648772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.649535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.649876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.649975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.650367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.656142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.656529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.657283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.657722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.742902 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lazy-loading 'services' on Instance uuid 3fc5dcaa-3275-47eb-88ec-62b23505c91b {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:28.788598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 HTTP/1.1 May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:28.790698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:28.792146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '621e54b1-cdde-493f-9bad-1c1222a0731a', 'id': '83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07'} May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a in vm_state rescued. Cannot detach_volume while the instance is in this state. May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.792889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.794038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 409 to user: Cannot 'detach_volume' instance 621e54b1-cdde-493f-9bad-1c1222a0731a while it is in vm_state rescued {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.795044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f893a4dd-3a47-41ab-a440-e30de4ec2ccb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 409 len: 152 microversion: 2.1 time: 0.146097 May 25 16:32:28.795044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 478/931] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 152 bytes in 147 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 25 16:32:28.799195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.799846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.806272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.806719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.807550 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.808130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.820404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e999b8dc-6a65-4189-acde-482e59b825d5) May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Request method failure captured: May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861 HTTP/1.1 May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlcTScjWaWhWZ_Y2deTS_iB2n7fhEPm9xpcRCdPE-r6ZMxY0ZuDpSToYfeI_E_5xfVHNIs52uYE6eqLoLyVrCNCdQhvtSm-zE7RKP38WrqQwdjdKcvzjavXk0OYwhbdCOQ2Ef36er8czPOW__xeJxnmKi5Ep7buFXy2Oju0Jbg9THQA May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:28.822776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_073368e447af465abb41aa556ee46dc9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/073368e447af465abb41aa556ee46dc9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: b53b45a485b14e0b9273c019dc9ec206 May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:32:28.824173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e999b8dc-6a65-4189-acde-482e59b825d5) May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': '3fc5dcaa-3275-47eb-88ec-62b23505c91b', 'id': 'ca477fe6-273b-4b1f-a801-7dc44a982861'} May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.824783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e999b8dc-6a65-4189-acde-482e59b825d5) May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.825825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e999b8dc-6a65-4189-acde-482e59b825d5) May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.826955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e999b8dc-6a65-4189-acde-482e59b825d5) May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.827997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e999b8dc-6a65-4189-acde-482e59b825d5) May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e999b8dc-6a65-4189-acde-482e59b825d5) May 25 16:32:28.829012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.830003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e999b8dc-6a65-4189-acde-482e59b825d5) {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.830003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9cf477c1-37b1-4aa9-a59d-95f421d360e9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861" status: 400 len: 269 microversion: 2.1 time: 0.187197 May 25 16:32:28.830003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 454/932] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861 => generated 269 bytes in 188 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:28.830845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-752fbd13-7e7a-48fc-a19f-69044c923e2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.832008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-752fbd13-7e7a-48fc-a19f-69044c923e2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.840554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-752fbd13-7e7a-48fc-a19f-69044c923e2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.840998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-752fbd13-7e7a-48fc-a19f-69044c923e2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.841755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-752fbd13-7e7a-48fc-a19f-69044c923e2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.842289 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-752fbd13-7e7a-48fc-a19f-69044c923e2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.876737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-752fbd13-7e7a-48fc-a19f-69044c923e2a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050380 May 25 16:32:28.877528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 455/933] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:28.882288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-080b115a-f0b1-46a0-a94b-87acaf915a6c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.884011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-080b115a-f0b1-46a0-a94b-87acaf915a6c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.891054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-080b115a-f0b1-46a0-a94b-87acaf915a6c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.891527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-080b115a-f0b1-46a0-a94b-87acaf915a6c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.892271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-080b115a-f0b1-46a0-a94b-87acaf915a6c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.892742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-080b115a-f0b1-46a0-a94b-87acaf915a6c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.893512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:28.925020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-080b115a-f0b1-46a0-a94b-87acaf915a6c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046652 May 25 16:32:28.925655 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 456/934] 10.222.0.45 () {60 vars in 1342 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:32:28.930053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.930750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.938067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.938509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.939304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.939728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.952724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4021448e-3d16-4add-b1e2-44e917c6bf5b) May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:28.954891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:28.956492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4021448e-3d16-4add-b1e2-44e917c6bf5b) May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.957387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4021448e-3d16-4add-b1e2-44e917c6bf5b) May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:28.958584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4021448e-3d16-4add-b1e2-44e917c6bf5b) May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.959816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4021448e-3d16-4add-b1e2-44e917c6bf5b) May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.960820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4021448e-3d16-4add-b1e2-44e917c6bf5b) May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4021448e-3d16-4add-b1e2-44e917c6bf5b) May 25 16:32:28.961923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:28.962921 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] HTTP exception thrown: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.963003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4021448e-3d16-4add-b1e2-44e917c6bf5b) {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.963003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1c8e7b4c-6537-4fed-874e-a2bf74968443 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 0.162448 May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Request method failure captured: May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 HTTP/1.1 May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbWFeoxbZ3XsqIpF0DsfOI5naE8sdWXPvt2-XBM059bd-3IcPRs2zKgCatH24oxhBCqAGB9S8-2TnjMb2HOQB8fgsaN7fMFsDRgpSeLztlAjjNXbpOLdf3DjKzT0d5VX9i7usk6EiBa_tJtHYRJkUINtSZbIduMdV6TxGU5Onxayw4 May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:28.963193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 7da83fc4029c4cdebca07c7247669bd0 May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.964730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'b3b920a7-fa28-4150-b889-d9a2ea63bc16'} May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:28.965490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:28.966690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:28.967861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Returning 404 to user: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:28.967861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fc3774dc-5310-464c-aa23-5774ecf3c6f0 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 404 len: 111 microversion: 2.1 time: 0.031722 May 25 16:32:28.967861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 457/935] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:32:28.968073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 479/936] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 269 bytes in 165 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:28.968073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.968073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.971829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7be9112c-47a5-4eb2-99e8-6dd469f8d1d6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:28.972295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7be9112c-47a5-4eb2-99e8-6dd469f8d1d6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:28.973466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.974268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.975208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.975638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.980125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7be9112c-47a5-4eb2-99e8-6dd469f8d1d6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.980784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7be9112c-47a5-4eb2-99e8-6dd469f8d1d6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:28.981725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7be9112c-47a5-4eb2-99e8-6dd469f8d1d6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:28.982682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7be9112c-47a5-4eb2-99e8-6dd469f8d1d6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:29.014903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7be9112c-47a5-4eb2-99e8-6dd469f8d1d6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips" status: 200 len: 146 microversion: 2.1 time: 0.055068 May 25 16:32:29.015678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 458/937] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:32:28 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a/ips => generated 146 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:29.026050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f432c839-fa38-46ad-a53d-7412011626cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:32:29.026669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f432c839-fa38-46ad-a53d-7412011626cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:29.033232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f432c839-fa38-46ad-a53d-7412011626cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:29.033993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f432c839-fa38-46ad-a53d-7412011626cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:29.034961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f432c839-fa38-46ad-a53d-7412011626cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:29.035433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f432c839-fa38-46ad-a53d-7412011626cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:29.068302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lazy-loading 'services' on Instance uuid 3fc5dcaa-3275-47eb-88ec-62b23505c91b {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:29.117389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f432c839-fa38-46ad-a53d-7412011626cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/action" status: 202 len: 0 microversion: 2.1 time: 0.095288 May 25 16:32:29.121164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 459/938] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:32:29 2022] POST /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/action => generated 0 bytes in 97 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:32:29.125092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:29.125696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:29.134079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:29.134079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:29.134079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:29.134473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:29.344509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dfaae16-e5f4-427d-96da-3ca9184b3c8a) May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Request method failure captured: May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861 HTTP/1.1 May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlcTScjWaWhWZ_Y2deTS_iB2n7fhEPm9xpcRCdPE-r6ZMxY0ZuDpSToYfeI_E_5xfVHNIs52uYE6eqLoLyVrCNCdQhvtSm-zE7RKP38WrqQwdjdKcvzjavXk0OYwhbdCOQ2Ef36er8czPOW__xeJxnmKi5Ep7buFXy2Oju0Jbg9THQA May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:29.348524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_073368e447af465abb41aa556ee46dc9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/073368e447af465abb41aa556ee46dc9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/073368e447af465abb41aa556ee46dc9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 073368e447af465abb41aa556ee46dc9 May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1046884794 May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: b53b45a485b14e0b9273c019dc9ec206 May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-VolumesSnapshotTestJSON-1046884794-project May 25 16:32:29.350630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dfaae16-e5f4-427d-96da-3ca9184b3c8a) May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'server_id': '3fc5dcaa-3275-47eb-88ec-62b23505c91b', 'id': 'ca477fe6-273b-4b1f-a801-7dc44a982861'} May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:29.352011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dfaae16-e5f4-427d-96da-3ca9184b3c8a) May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:29.353192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dfaae16-e5f4-427d-96da-3ca9184b3c8a) May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.355122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dfaae16-e5f4-427d-96da-3ca9184b3c8a) May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:29.360823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dfaae16-e5f4-427d-96da-3ca9184b3c8a) May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dfaae16-e5f4-427d-96da-3ca9184b3c8a) May 25 16:32:29.366611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:29.369594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dfaae16-e5f4-427d-96da-3ca9184b3c8a) {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:29.369594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d669195d-af9e-4f90-a4fa-440ed3e774d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861" status: 400 len: 269 microversion: 2.1 time: 0.404077 May 25 16:32:29.369594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 480/939] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:28 2022] DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b/os-volume_attachments/ca477fe6-273b-4b1f-a801-7dc44a982861 => generated 269 bytes in 405 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:29.376584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1026335801", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-2139248485", "security_groups": [{"name": "tempest-secgroup-smoke-170858155"}], "block_device_mapping_v2": [{"uuid": "b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "7fd1aa0d-38cb-4e70-ab28-67c3630862cc"}]}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:32:29.417361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lazy-loading 'services' on Instance uuid d86354dc-f3d4-409d-b331-9f708947be2a {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:32:29.598097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ca5383-2e28-499a-9340-ac238149f035) May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Request method failure captured: May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 HTTP/1.1 May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbxSECf_FI-Mow8-n_dzdcjtCSG6DjkSu9TcMj7ujJf71u6HtldGBMK6Rd5DZpWOLvNMF_GUE0LnnpaI6ycafoHNW7d14JACqyND-3e6bKB3cmRZk16XPd8zoLIpayEiuRTL-LM4uHnUEMd4U2DdcRySoAGiwEczVpGjVZTCQWl1P0 May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:29.600641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/711b13e4ea3e4269a0082811f6708d87"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 711b13e4ea3e4269a0082811f6708d87 May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1349570600 May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: e40faab6a34f4d368462cdcafb7bcfea May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestMinimumBasicScenario-1349570600-project May 25 16:32:29.602059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ca5383-2e28-499a-9340-ac238149f035) May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'server_id': 'd86354dc-f3d4-409d-b331-9f708947be2a', 'id': '8b19afac-6e47-40fb-a1df-82a200122a51'} May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:29.605077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ca5383-2e28-499a-9340-ac238149f035) May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:29.606062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ca5383-2e28-499a-9340-ac238149f035) May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.607265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ca5383-2e28-499a-9340-ac238149f035) May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:29.608251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ca5383-2e28-499a-9340-ac238149f035) May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ca5383-2e28-499a-9340-ac238149f035) May 25 16:32:29.609312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:29.610557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ca5383-2e28-499a-9340-ac238149f035) {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:29.610557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d6cfcd72-b03b-4c1c-ab46-e9125ad0eff4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51" status: 400 len: 269 microversion: 2.1 time: 0.481227 May 25 16:32:29.610557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 460/940] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:32:29 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a/os-volume_attachments/8b19afac-6e47-40fb-a1df-82a200122a51 => generated 269 bytes in 483 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:29.610557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2761c8c0-be98-48e7-b1f9-2319fb57f0d1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:29.610894 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2761c8c0-be98-48e7-b1f9-2319fb57f0d1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:29.614804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2761c8c0-be98-48e7-b1f9-2319fb57f0d1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:29.615288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2761c8c0-be98-48e7-b1f9-2319fb57f0d1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:29.616056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2761c8c0-be98-48e7-b1f9-2319fb57f0d1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:29.616490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2761c8c0-be98-48e7-b1f9-2319fb57f0d1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:29.702848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] validate_networks() for [('7fd1aa0d-38cb-4e70-ab28-67c3630862cc', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:32:29.922532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2761c8c0-be98-48e7-b1f9-2319fb57f0d1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:32:29.933087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2761c8c0-be98-48e7-b1f9-2319fb57f0d1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1724 microversion: 2.1 time: 0.329958 May 25 16:32:29.933677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 461/941] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:29 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1724 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:32:29.938282 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-135fcfa3-0fd2-4260-b659-3120f34441c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:32:29.939283 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-135fcfa3-0fd2-4260-b659-3120f34441c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:29.945777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-135fcfa3-0fd2-4260-b659-3120f34441c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:29.951283 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-135fcfa3-0fd2-4260-b659-3120f34441c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:29.952064 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-135fcfa3-0fd2-4260-b659-3120f34441c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:29.952503 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-135fcfa3-0fd2-4260-b659-3120f34441c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:30.030865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-135fcfa3-0fd2-4260-b659-3120f34441c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/action" status: 200 len: 41363 microversion: 2.1 time: 0.095920 May 25 16:32:30.032514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 462/942] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:32:29 2022] POST /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7/action => generated 41363 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:32:30.037444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cd4392ab-8a9b-4462-973e-d8363cfcf3c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:32:30.039176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cd4392ab-8a9b-4462-973e-d8363cfcf3c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:30.047940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cd4392ab-8a9b-4462-973e-d8363cfcf3c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:30.048599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cd4392ab-8a9b-4462-973e-d8363cfcf3c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:30.049660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cd4392ab-8a9b-4462-973e-d8363cfcf3c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:30.050200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cd4392ab-8a9b-4462-973e-d8363cfcf3c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:30.271342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cd4392ab-8a9b-4462-973e-d8363cfcf3c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "POST /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/action" status: 200 len: 38015 microversion: 2.20 time: 0.238106 May 25 16:32:30.271942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 463/943] 10.222.0.45 () {62 vars in 1318 bytes} [Wed May 25 16:32:30 2022] POST /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4/action => generated 38015 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 361 bytes (1 switches on core 0) May 25 16:32:30.402115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:32:30.402637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:32:30.403230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:32:30.403750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:32:30.404251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:32:30.418702 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:30.419477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:30.420751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:30.421422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:30.444448 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Getting quotas for project 01367a52e9104f85baec4b18ef8bf258. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:32:30.448762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Getting quotas for user 7da83fc4029c4cdebca07c7247669bd0 and project 01367a52e9104f85baec4b18ef8bf258. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:32:30.517189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:32:30.519333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b32b72af-51ec-4819-a3e9-0ee820a86062] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79',volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:32:30.661901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Volume b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79 status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-aa3f0347-2cc0-4ebd-a378-99dfa87ba539) May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Request method failure captured: May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: POST /compute/v2.1/servers HTTP/1.1 May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 477 May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlbWFeoxbZ3XsqIpF0DsfOI5naE8sdWXPvt2-XBM059bd-3IcPRs2zKgCatH24oxhBCqAGB9S8-2TnjMb2HOQB8fgsaN7fMFsDRgpSeLztlAjjNXbpOLdf3DjKzT0d5VX9i7usk6EiBa_tJtHYRJkUINtSZbIduMdV6TxGU5Onxayw4 May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:30.668395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 7da83fc4029c4cdebca07c7247669bd0 May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:30.669948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {"server": {"name": "tempest-TestVolumeBootPattern-server-1026335801", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-2139248485", "security_groups": [{"name": "tempest-secgroup-smoke-170858155"}], "block_device_mapping_v2": [{"uuid": "b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "7fd1aa0d-38cb-4e70-ab28-67c3630862cc"}]}} May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Invalid volume: Invalid input received: Invalid volume: Volume b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79 status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-aa3f0347-2cc0-4ebd-a378-99dfa87ba539) May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'body': {'server': {'name': 'tempest-TestVolumeBootPattern-server-1026335801', 'imageRef': '', 'flavorRef': '452', 'key_name': 'tempest-TestVolumeBootPattern-2139248485', 'security_groups': [{'name': 'tempest-secgroup-smoke-170858155'}], 'block_device_mapping_v2': [{'source_type': 'volume', 'destination_type': 'volume', 'boot_index': 0, 'delete_on_termination': False, 'volume_id': 'b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79'}], 'networks': [{'uuid': '7fd1aa0d-38cb-4e70-ab28-67c3630862cc'}]}}} May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:30.672985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 808, in attachment_create May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi LOG.error('Create attachment failed for volume ' May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.force_reraise() May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise self.value May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 800, in attachment_create May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi attachment_ref = cinderclient(context, '3.44').attachments.create( May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/api_versions.py", line 421, in substitution May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method.func(obj, *args, **kwargs) May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/attachments.py", line 39, in create May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi retval = self._create('/attachments', body, 'attachment') May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 306, in _create May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:30.673745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Volume b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79 status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-aa3f0347-2cc0-4ebd-a378-99dfa87ba539) May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1914, in _validate_bdm May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._check_attach_and_reserve_volume( May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4858, in _check_attach_and_reserve_volume May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi attachment_id = self.volume_api.attachment_create( May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:32:30.675020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 808, in attachment_create May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi LOG.error('Create attachment failed for volume ' May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.force_reraise() May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise self.value May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 800, in attachment_create May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi attachment_ref = cinderclient(context, '3.44').attachments.create( May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/api_versions.py", line 421, in substitution May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method.func(obj, *args, **kwargs) May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/attachments.py", line 39, in create May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi retval = self._create('/attachments', body, 'attachment') May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 306, in _create May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 25 16:32:30.676312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Volume b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79 status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-aa3f0347-2cc0-4ebd-a378-99dfa87ba539) May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2149, in create May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._create_instance( May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1676, in _create_instance May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1536, in _provision_instances May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._cleanup_build_artifacts(None, instances_to_build) May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 16:32:30.677542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.force_reraise() May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise self.value May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1446, in _provision_instances May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._bdm_validate_set_size_and_instance(context, May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1809, in _bdm_validate_set_size_and_instance May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._validate_bdm( May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1923, in _validate_bdm May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Volume b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79 status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-aa3f0347-2cc0-4ebd-a378-99dfa87ba539) May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:30.678855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Volume b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79 status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-aa3f0347-2cc0-4ebd-a378-99dfa87ba539) May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:30.680308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:30.681780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Volume b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79 status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-aa3f0347-2cc0-4ebd-a378-99dfa87ba539) May 25 16:32:30.681780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:30.681780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Volume b0ed48a0-84c0-4544-8bbc-4ec0a70f4c79 status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-aa3f0347-2cc0-4ebd-a378-99dfa87ba539) {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:30.681780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-98dd2898-1513-43eb-b3fa-74d81b52126d tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 400 len: 295 microversion: 2.1 time: 1.304190 May 25 16:32:30.681780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 481/944] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:32:29 2022] POST /compute/v2.1/servers => generated 295 bytes in 1305 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 25 16:32:30.950565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-173a591c-7cac-4b93-bc36-2c63efa7fa24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:30.951286 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-173a591c-7cac-4b93-bc36-2c63efa7fa24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:30.958117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-173a591c-7cac-4b93-bc36-2c63efa7fa24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:30.958565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-173a591c-7cac-4b93-bc36-2c63efa7fa24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:30.959364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-173a591c-7cac-4b93-bc36-2c63efa7fa24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:30.959795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-173a591c-7cac-4b93-bc36-2c63efa7fa24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:31.207400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-173a591c-7cac-4b93-bc36-2c63efa7fa24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:32:31.215637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-173a591c-7cac-4b93-bc36-2c63efa7fa24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1724 microversion: 2.1 time: 0.269238 May 25 16:32:31.216209 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 464/945] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:30 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1724 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:32:31.225004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:31.225504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:31.232791 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:31.233258 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:31.233991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:31.234431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:31.247754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] HTTP exception thrown: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Request method failure captured: May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 HTTP/1.1 May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlbWFeoxbZ3XsqIpF0DsfOI5naE8sdWXPvt2-XBM059bd-3IcPRs2zKgCatH24oxhBCqAGB9S8-2TnjMb2HOQB8fgsaN7fMFsDRgpSeLztlAjjNXbpOLdf3DjKzT0d5VX9i7usk6EiBa_tJtHYRJkUINtSZbIduMdV6TxGU5Onxayw4 May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:32:31.249305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 7da83fc4029c4cdebca07c7247669bd0 May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.250801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'b3b920a7-fa28-4150-b889-d9a2ea63bc16'} May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:31.251850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.253322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:32:31.254837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Returning 404 to user: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:31.254837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f5c30e74-5e5c-4acf-a082-03528ced81d3 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "DELETE /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 404 len: 111 microversion: 2.1 time: 0.029746 May 25 16:32:31.254837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 482/946] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:32:31 2022] DELETE /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:32:31.259419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:31.260035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] [instance: b3b920a7-fa28-4150-b889-d9a2ea63bc16] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:31.266811 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:31.267333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:31.268094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:31.268535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:31.282379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] HTTP exception thrown: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Request method failure captured: May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 HTTP/1.1 May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlbWFeoxbZ3XsqIpF0DsfOI5naE8sdWXPvt2-XBM059bd-3IcPRs2zKgCatH24oxhBCqAGB9S8-2TnjMb2HOQB8fgsaN7fMFsDRgpSeLztlAjjNXbpOLdf3DjKzT0d5VX9i7usk6EiBa_tJtHYRJkUINtSZbIduMdV6TxGU5Onxayw4 May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:32:31.284177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/01367a52e9104f85baec4b18ef8bf258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: 01367a52e9104f85baec4b18ef8bf258 May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-TestVolumeBootPattern-1840251582 May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 7da83fc4029c4cdebca07c7247669bd0 May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-TestVolumeBootPattern-1840251582-project May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.285711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'b3b920a7-fa28-4150-b889-d9a2ea63bc16'} May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:32:31.286469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. May 25 16:32:31.287690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:32:31.288715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Returning 404 to user: Instance b3b920a7-fa28-4150-b889-d9a2ea63bc16 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:32:31.288715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7a074a6a-14f0-416c-a2a1-e03f05cfade9 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16" status: 404 len: 111 microversion: 2.1 time: 0.029913 May 25 16:32:31.288715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 465/947] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:31 2022] GET /compute/v2.1/servers/b3b920a7-fa28-4150-b889-d9a2ea63bc16 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:32:32.233380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-975fd360-9e56-4d46-86fd-23d31846ce2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:32.234324 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-975fd360-9e56-4d46-86fd-23d31846ce2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:32.241705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-975fd360-9e56-4d46-86fd-23d31846ce2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:32.242168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-975fd360-9e56-4d46-86fd-23d31846ce2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:32.242945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-975fd360-9e56-4d46-86fd-23d31846ce2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:32.243408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-975fd360-9e56-4d46-86fd-23d31846ce2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:32.375851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-975fd360-9e56-4d46-86fd-23d31846ce2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:32:32.384273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-975fd360-9e56-4d46-86fd-23d31846ce2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1724 microversion: 2.1 time: 0.155186 May 25 16:32:32.384844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 483/948] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:32 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1724 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:32:33.405290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4878e618-53ff-4318-84f0-3907510917ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:33.405949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4878e618-53ff-4318-84f0-3907510917ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:33.414486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4878e618-53ff-4318-84f0-3907510917ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:33.415051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4878e618-53ff-4318-84f0-3907510917ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:33.416374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4878e618-53ff-4318-84f0-3907510917ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:33.416933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4878e618-53ff-4318-84f0-3907510917ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:33.564115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4878e618-53ff-4318-84f0-3907510917ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:32:33.572171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4878e618-53ff-4318-84f0-3907510917ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1730 microversion: 2.1 time: 0.172709 May 25 16:32:33.572740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 466/949] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:32:33 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1730 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:32:33.590840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a4b2d578-ee5e-4070-bfc2-ad2661e95f52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:32:33.591471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a4b2d578-ee5e-4070-bfc2-ad2661e95f52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:32:33.598245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4b2d578-ee5e-4070-bfc2-ad2661e95f52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:33.598700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4b2d578-ee5e-4070-bfc2-ad2661e95f52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:33.599492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4b2d578-ee5e-4070-bfc2-ad2661e95f52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:33.599932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4b2d578-ee5e-4070-bfc2-ad2661e95f52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:33.677309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a4b2d578-ee5e-4070-bfc2-ad2661e95f52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.092234 May 25 16:32:33.677921 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 484/950] 10.222.0.45 () {58 vars in 1274 bytes} [Wed May 25 16:32:33 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-interface => generated 285 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:34.655744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-e7874c70-b625-40c7-a55b-4f8bc59db5a8 req-a4e7c809-c86d-4b67-9bd5-96f5064d64de service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "621e54b1-cdde-493f-9bad-1c1222a0731a", "tag": "9b36da50-ea60-4505-bbbd-48a647e0698b"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:32:34.664146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-e7874c70-b625-40c7-a55b-4f8bc59db5a8 req-a4e7c809-c86d-4b67-9bd5-96f5064d64de service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:34.664730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-e7874c70-b625-40c7-a55b-4f8bc59db5a8 req-a4e7c809-c86d-4b67-9bd5-96f5064d64de service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:34.676356 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-e7874c70-b625-40c7-a55b-4f8bc59db5a8 req-a4e7c809-c86d-4b67-9bd5-96f5064d64de service nova] Creating event network-changed:9b36da50-ea60-4505-bbbd-48a647e0698b for instance 621e54b1-cdde-493f-9bad-1c1222a0731a on hv-cinder-84327 May 25 16:32:34.681630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-e7874c70-b625-40c7-a55b-4f8bc59db5a8 req-a4e7c809-c86d-4b67-9bd5-96f5064d64de service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.103361 May 25 16:32:34.682394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 467/951] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:32:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:32:36.699004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-c503f2e5-848f-41e3-ad26-e16dfe91409b req-cf8db14d-2154-4fb4-ba8f-335b339152d8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "621e54b1-cdde-493f-9bad-1c1222a0731a", "tag": "9b36da50-ea60-4505-bbbd-48a647e0698b"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:32:36.707457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-c503f2e5-848f-41e3-ad26-e16dfe91409b req-cf8db14d-2154-4fb4-ba8f-335b339152d8 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:32:36.707893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-c503f2e5-848f-41e3-ad26-e16dfe91409b req-cf8db14d-2154-4fb4-ba8f-335b339152d8 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:32:36.719075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-c503f2e5-848f-41e3-ad26-e16dfe91409b req-cf8db14d-2154-4fb4-ba8f-335b339152d8 service nova] Creating event network-changed:9b36da50-ea60-4505-bbbd-48a647e0698b for instance 621e54b1-cdde-493f-9bad-1c1222a0731a on hv-cinder-84327 May 25 16:32:36.725506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-c503f2e5-848f-41e3-ad26-e16dfe91409b req-cf8db14d-2154-4fb4-ba8f-335b339152d8 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030640 May 25 16:32:36.726079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 485/952] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:32:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:02.589605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=62 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:02.590386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=76,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:02.595175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6e5db5ca-2ab1-461a-92f9-82acc53b99f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:02.595862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6e5db5ca-2ab1-461a-92f9-82acc53b99f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:02.603525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e5db5ca-2ab1-461a-92f9-82acc53b99f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:02.604014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e5db5ca-2ab1-461a-92f9-82acc53b99f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:02.604751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e5db5ca-2ab1-461a-92f9-82acc53b99f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:02.605192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e5db5ca-2ab1-461a-92f9-82acc53b99f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:02.693542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-6e5db5ca-2ab1-461a-92f9-82acc53b99f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lazy-loading 'services' on Instance uuid 621e54b1-cdde-493f-9bad-1c1222a0731a {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:33:02.794995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6e5db5ca-2ab1-461a-92f9-82acc53b99f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07" status: 202 len: 0 microversion: 2.1 time: 0.206537 May 25 16:33:02.795860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 468/953] 10.222.0.45 () {58 vars in 1415 bytes} [Wed May 25 16:33:02 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments/83fa7e3b-64e6-4a0a-a790-7b0be3ff8a07 => generated 0 bytes in 209 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:33:02.805945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:02.807110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=6 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:02.807827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=70 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:02.808572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=58 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:02.813290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-38fb9936-8b2a-4581-b078-97453f8a8832 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:02.814388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-38fb9936-8b2a-4581-b078-97453f8a8832 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:02.823793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38fb9936-8b2a-4581-b078-97453f8a8832 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:02.824292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38fb9936-8b2a-4581-b078-97453f8a8832 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:02.825108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38fb9936-8b2a-4581-b078-97453f8a8832 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:02.825542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38fb9936-8b2a-4581-b078-97453f8a8832 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:02.857271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-38fb9936-8b2a-4581-b078-97453f8a8832 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054298 May 25 16:33:02.857848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 486/954] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:02 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:03.874808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4d7a8445-b85f-4653-9853-4e494ec872c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:03.876562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4d7a8445-b85f-4653-9853-4e494ec872c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:03.886305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d7a8445-b85f-4653-9853-4e494ec872c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:03.886778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d7a8445-b85f-4653-9853-4e494ec872c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:03.887573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d7a8445-b85f-4653-9853-4e494ec872c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:03.888002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d7a8445-b85f-4653-9853-4e494ec872c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:03.919428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4d7a8445-b85f-4653-9853-4e494ec872c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052556 May 25 16:33:03.919987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 469/955] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:03 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:04.937128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3f9dc41c-3b0a-4ca4-bdc4-646d98c39c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:04.938648 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3f9dc41c-3b0a-4ca4-bdc4-646d98c39c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:04.951666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3f9dc41c-3b0a-4ca4-bdc4-646d98c39c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:04.952408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3f9dc41c-3b0a-4ca4-bdc4-646d98c39c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:04.953661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3f9dc41c-3b0a-4ca4-bdc4-646d98c39c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:04.954381 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3f9dc41c-3b0a-4ca4-bdc4-646d98c39c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:04.987960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3f9dc41c-3b0a-4ca4-bdc4-646d98c39c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057673 May 25 16:33:04.988546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 487/956] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:04 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:06.003473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fbacf2e9-f8d4-4052-82af-10197632f9fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:06.004619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fbacf2e9-f8d4-4052-82af-10197632f9fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:06.013040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fbacf2e9-f8d4-4052-82af-10197632f9fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:06.013488 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fbacf2e9-f8d4-4052-82af-10197632f9fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:06.014255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fbacf2e9-f8d4-4052-82af-10197632f9fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:06.014653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fbacf2e9-f8d4-4052-82af-10197632f9fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:06.047015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fbacf2e9-f8d4-4052-82af-10197632f9fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050135 May 25 16:33:06.047621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 470/957] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:05 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:07.058756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0582aab6-bfc7-45fc-8c66-5ceb6d6c1fd9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:07.059932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0582aab6-bfc7-45fc-8c66-5ceb6d6c1fd9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:07.066983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0582aab6-bfc7-45fc-8c66-5ceb6d6c1fd9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:07.067468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0582aab6-bfc7-45fc-8c66-5ceb6d6c1fd9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:07.068248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0582aab6-bfc7-45fc-8c66-5ceb6d6c1fd9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:07.068701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0582aab6-bfc7-45fc-8c66-5ceb6d6c1fd9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:07.099179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0582aab6-bfc7-45fc-8c66-5ceb6d6c1fd9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044859 May 25 16:33:07.099785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 488/958] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:07 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:08.111027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-497d7a7a-975f-44b0-8bf2-af08c8d73d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:08.111983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-497d7a7a-975f-44b0-8bf2-af08c8d73d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:08.118611 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-497d7a7a-975f-44b0-8bf2-af08c8d73d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:08.119134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-497d7a7a-975f-44b0-8bf2-af08c8d73d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:08.119892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-497d7a7a-975f-44b0-8bf2-af08c8d73d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:08.120336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-497d7a7a-975f-44b0-8bf2-af08c8d73d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:08.150589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-497d7a7a-975f-44b0-8bf2-af08c8d73d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043568 May 25 16:33:08.151189 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 471/959] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:08 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:09.165626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6f467bec-dda5-4afb-ba74-91cb66e387e9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:09.166608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6f467bec-dda5-4afb-ba74-91cb66e387e9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:09.173449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f467bec-dda5-4afb-ba74-91cb66e387e9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:09.173902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f467bec-dda5-4afb-ba74-91cb66e387e9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:09.174671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f467bec-dda5-4afb-ba74-91cb66e387e9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:09.175091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f467bec-dda5-4afb-ba74-91cb66e387e9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:09.205147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6f467bec-dda5-4afb-ba74-91cb66e387e9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043859 May 25 16:33:09.205700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 489/960] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:09 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:10.217322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3b157829-c7d7-4d03-a462-8600c0300ba8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:10.218252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3b157829-c7d7-4d03-a462-8600c0300ba8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:10.225390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3b157829-c7d7-4d03-a462-8600c0300ba8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:10.225876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3b157829-c7d7-4d03-a462-8600c0300ba8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:10.226639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3b157829-c7d7-4d03-a462-8600c0300ba8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:10.227076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3b157829-c7d7-4d03-a462-8600c0300ba8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:10.268007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3b157829-c7d7-4d03-a462-8600c0300ba8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054161 May 25 16:33:10.268781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 472/961] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:10 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:11.284753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e172614d-86bd-43b4-a67b-1e240c3510df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:11.285995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e172614d-86bd-43b4-a67b-1e240c3510df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:11.294930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e172614d-86bd-43b4-a67b-1e240c3510df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:11.295410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e172614d-86bd-43b4-a67b-1e240c3510df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:11.296175 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e172614d-86bd-43b4-a67b-1e240c3510df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:11.296611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e172614d-86bd-43b4-a67b-1e240c3510df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:11.327130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e172614d-86bd-43b4-a67b-1e240c3510df tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048905 May 25 16:33:11.327657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 490/962] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:11 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:12.338572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cfab5917-d3c6-4da3-9848-4f361ad52c6e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:12.339603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cfab5917-d3c6-4da3-9848-4f361ad52c6e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:12.346352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfab5917-d3c6-4da3-9848-4f361ad52c6e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:12.346800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfab5917-d3c6-4da3-9848-4f361ad52c6e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:12.347586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfab5917-d3c6-4da3-9848-4f361ad52c6e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:12.348018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfab5917-d3c6-4da3-9848-4f361ad52c6e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:12.378381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cfab5917-d3c6-4da3-9848-4f361ad52c6e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044046 May 25 16:33:12.378944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 473/963] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:12 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:13.390211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-95356a66-72cc-4a2c-ad43-9f84089b5bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:13.391180 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-95356a66-72cc-4a2c-ad43-9f84089b5bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:13.398154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-95356a66-72cc-4a2c-ad43-9f84089b5bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:13.398602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-95356a66-72cc-4a2c-ad43-9f84089b5bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:13.399410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-95356a66-72cc-4a2c-ad43-9f84089b5bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:13.399841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-95356a66-72cc-4a2c-ad43-9f84089b5bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:13.430836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-95356a66-72cc-4a2c-ad43-9f84089b5bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044688 May 25 16:33:13.431463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 491/964] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:13 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:14.444219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2966766b-9a05-4256-b1b6-a047c9e8fba9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:14.445159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2966766b-9a05-4256-b1b6-a047c9e8fba9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:14.452376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2966766b-9a05-4256-b1b6-a047c9e8fba9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:14.452826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2966766b-9a05-4256-b1b6-a047c9e8fba9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:14.453591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2966766b-9a05-4256-b1b6-a047c9e8fba9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:14.454026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2966766b-9a05-4256-b1b6-a047c9e8fba9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:14.483602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2966766b-9a05-4256-b1b6-a047c9e8fba9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043295 May 25 16:33:14.484150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 474/965] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:14 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:15.496143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b22b3da4-0b0a-4785-a68d-bade18f394f9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:15.497124 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b22b3da4-0b0a-4785-a68d-bade18f394f9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:15.503899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b22b3da4-0b0a-4785-a68d-bade18f394f9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:15.504353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b22b3da4-0b0a-4785-a68d-bade18f394f9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:15.505121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b22b3da4-0b0a-4785-a68d-bade18f394f9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:15.505552 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b22b3da4-0b0a-4785-a68d-bade18f394f9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:15.534872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b22b3da4-0b0a-4785-a68d-bade18f394f9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042753 May 25 16:33:15.535482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 492/966] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:15 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:16.546856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-62f84523-aa9b-4389-90e5-0f9bfb20c05c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:16.547810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-62f84523-aa9b-4389-90e5-0f9bfb20c05c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:16.554457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62f84523-aa9b-4389-90e5-0f9bfb20c05c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:16.554903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62f84523-aa9b-4389-90e5-0f9bfb20c05c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:16.555682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62f84523-aa9b-4389-90e5-0f9bfb20c05c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:16.556114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62f84523-aa9b-4389-90e5-0f9bfb20c05c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:16.585330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-62f84523-aa9b-4389-90e5-0f9bfb20c05c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042398 May 25 16:33:16.585885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 475/967] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:16 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:17.597264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4b5bbfa3-d894-4abe-ba62-ff2248f4d911 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:17.598198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4b5bbfa3-d894-4abe-ba62-ff2248f4d911 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:17.604527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4b5bbfa3-d894-4abe-ba62-ff2248f4d911 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:17.605031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4b5bbfa3-d894-4abe-ba62-ff2248f4d911 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:17.605798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4b5bbfa3-d894-4abe-ba62-ff2248f4d911 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:17.606232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4b5bbfa3-d894-4abe-ba62-ff2248f4d911 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:17.636716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4b5bbfa3-d894-4abe-ba62-ff2248f4d911 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043565 May 25 16:33:17.637473 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 493/968] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:17 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:18.651740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9582df83-0afd-4053-9efd-d18307f45b91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:18.652644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9582df83-0afd-4053-9efd-d18307f45b91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:18.659658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9582df83-0afd-4053-9efd-d18307f45b91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:18.660114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9582df83-0afd-4053-9efd-d18307f45b91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:18.663063 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9582df83-0afd-4053-9efd-d18307f45b91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:18.663063 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9582df83-0afd-4053-9efd-d18307f45b91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:18.691797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9582df83-0afd-4053-9efd-d18307f45b91 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046044 May 25 16:33:18.692516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 476/969] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:18 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:19.705685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-142319d9-569f-47d2-aa7b-3eb362617943 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:19.706638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-142319d9-569f-47d2-aa7b-3eb362617943 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:19.714817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-142319d9-569f-47d2-aa7b-3eb362617943 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:19.715470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-142319d9-569f-47d2-aa7b-3eb362617943 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:19.716627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-142319d9-569f-47d2-aa7b-3eb362617943 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:19.717252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-142319d9-569f-47d2-aa7b-3eb362617943 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:19.751659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-142319d9-569f-47d2-aa7b-3eb362617943 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050423 May 25 16:33:19.755588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 494/970] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:19 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:20.767819 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e9a9eb5f-626a-49f3-88e1-c6f38e6afb7e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:20.769322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e9a9eb5f-626a-49f3-88e1-c6f38e6afb7e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:20.778958 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e9a9eb5f-626a-49f3-88e1-c6f38e6afb7e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:20.779446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e9a9eb5f-626a-49f3-88e1-c6f38e6afb7e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:20.780215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e9a9eb5f-626a-49f3-88e1-c6f38e6afb7e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:20.780654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e9a9eb5f-626a-49f3-88e1-c6f38e6afb7e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:20.810973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e9a9eb5f-626a-49f3-88e1-c6f38e6afb7e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049942 May 25 16:33:20.811561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 477/971] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:20 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:21.824984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4db35979-c6ab-4463-9d57-03b5bf5226b1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:21.826228 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4db35979-c6ab-4463-9d57-03b5bf5226b1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:21.834949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4db35979-c6ab-4463-9d57-03b5bf5226b1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:21.835560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4db35979-c6ab-4463-9d57-03b5bf5226b1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:21.836515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4db35979-c6ab-4463-9d57-03b5bf5226b1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:21.837032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4db35979-c6ab-4463-9d57-03b5bf5226b1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:21.870695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4db35979-c6ab-4463-9d57-03b5bf5226b1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051801 May 25 16:33:21.871280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 495/972] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:21 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:22.886713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d4cec88e-d4e6-4e7d-bda1-b4f2b7c1feb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:22.888550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d4cec88e-d4e6-4e7d-bda1-b4f2b7c1feb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:22.899454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4cec88e-d4e6-4e7d-bda1-b4f2b7c1feb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:22.899940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4cec88e-d4e6-4e7d-bda1-b4f2b7c1feb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:22.902713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4cec88e-d4e6-4e7d-bda1-b4f2b7c1feb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:22.902713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4cec88e-d4e6-4e7d-bda1-b4f2b7c1feb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:22.932009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d4cec88e-d4e6-4e7d-bda1-b4f2b7c1feb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052399 May 25 16:33:22.932609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 478/973] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:22 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:23.943671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-deb94f76-b415-4c90-9d92-fd8c38a0f94c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:23.944639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-deb94f76-b415-4c90-9d92-fd8c38a0f94c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:23.951764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-deb94f76-b415-4c90-9d92-fd8c38a0f94c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:23.952212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-deb94f76-b415-4c90-9d92-fd8c38a0f94c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:23.952936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-deb94f76-b415-4c90-9d92-fd8c38a0f94c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:23.953393 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-deb94f76-b415-4c90-9d92-fd8c38a0f94c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:23.983352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-deb94f76-b415-4c90-9d92-fd8c38a0f94c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043699 May 25 16:33:23.984129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 496/974] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:23 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:24.994917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ae6e97c3-01a9-4994-a8a5-3e04ad006146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:24.995879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ae6e97c3-01a9-4994-a8a5-3e04ad006146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:25.003817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae6e97c3-01a9-4994-a8a5-3e04ad006146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:25.004234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae6e97c3-01a9-4994-a8a5-3e04ad006146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:25.005215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae6e97c3-01a9-4994-a8a5-3e04ad006146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:25.005651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae6e97c3-01a9-4994-a8a5-3e04ad006146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:25.035963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ae6e97c3-01a9-4994-a8a5-3e04ad006146 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045067 May 25 16:33:25.036545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 479/975] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:24 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:26.048665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5826c674-87f3-4450-9100-f66bbf1f5afc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:26.049713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5826c674-87f3-4450-9100-f66bbf1f5afc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:26.057204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5826c674-87f3-4450-9100-f66bbf1f5afc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:26.057779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5826c674-87f3-4450-9100-f66bbf1f5afc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:26.058541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5826c674-87f3-4450-9100-f66bbf1f5afc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:26.058975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5826c674-87f3-4450-9100-f66bbf1f5afc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:26.091992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5826c674-87f3-4450-9100-f66bbf1f5afc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047741 May 25 16:33:26.092566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 497/976] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:26 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:27.226385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-094fc928-a377-4494-9042-af45bb666ac9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:27.227355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-094fc928-a377-4494-9042-af45bb666ac9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:27.233562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-094fc928-a377-4494-9042-af45bb666ac9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:27.234015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-094fc928-a377-4494-9042-af45bb666ac9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:27.234775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-094fc928-a377-4494-9042-af45bb666ac9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:27.235239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-094fc928-a377-4494-9042-af45bb666ac9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:27.264987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-094fc928-a377-4494-9042-af45bb666ac9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.164644 May 25 16:33:27.265482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 480/977] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:27 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:28.281184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c3d70c4c-ec56-4a94-935a-c62b7215a0ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:28.282630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c3d70c4c-ec56-4a94-935a-c62b7215a0ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:28.294122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3d70c4c-ec56-4a94-935a-c62b7215a0ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:28.294868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3d70c4c-ec56-4a94-935a-c62b7215a0ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:28.296167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3d70c4c-ec56-4a94-935a-c62b7215a0ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:28.296870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3d70c4c-ec56-4a94-935a-c62b7215a0ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:28.330812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c3d70c4c-ec56-4a94-935a-c62b7215a0ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056359 May 25 16:33:28.331553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 498/978] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:28 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:29.346415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f373248f-570b-43ee-a4dc-aa6a0c7509f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:29.348103 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f373248f-570b-43ee-a4dc-aa6a0c7509f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:29.357061 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f373248f-570b-43ee-a4dc-aa6a0c7509f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:29.357514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f373248f-570b-43ee-a4dc-aa6a0c7509f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:29.358278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f373248f-570b-43ee-a4dc-aa6a0c7509f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:29.358712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f373248f-570b-43ee-a4dc-aa6a0c7509f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:29.395995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f373248f-570b-43ee-a4dc-aa6a0c7509f8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056619 May 25 16:33:29.396558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 481/979] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:29 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:30.410465 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d4317f93-d8db-463f-9e72-7f7ee49a14b9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:30.411905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d4317f93-d8db-463f-9e72-7f7ee49a14b9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:30.423340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4317f93-d8db-463f-9e72-7f7ee49a14b9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:30.424152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4317f93-d8db-463f-9e72-7f7ee49a14b9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:30.425439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4317f93-d8db-463f-9e72-7f7ee49a14b9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:30.426150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4317f93-d8db-463f-9e72-7f7ee49a14b9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:30.464530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d4317f93-d8db-463f-9e72-7f7ee49a14b9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059345 May 25 16:33:30.465006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 499/980] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:30 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:31.480678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-960969da-90bb-4688-a009-8eff2a087802 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:31.481841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-960969da-90bb-4688-a009-8eff2a087802 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:31.490594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-960969da-90bb-4688-a009-8eff2a087802 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:31.491194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-960969da-90bb-4688-a009-8eff2a087802 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:31.492310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-960969da-90bb-4688-a009-8eff2a087802 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:31.492845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-960969da-90bb-4688-a009-8eff2a087802 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:31.527456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-960969da-90bb-4688-a009-8eff2a087802 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053562 May 25 16:33:31.528020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 482/981] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:31 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:32.541212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8fcefdda-6e37-45a8-907f-be4cb83f744e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:32.542241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8fcefdda-6e37-45a8-907f-be4cb83f744e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:32.549333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8fcefdda-6e37-45a8-907f-be4cb83f744e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:32.549786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8fcefdda-6e37-45a8-907f-be4cb83f744e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:32.550551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8fcefdda-6e37-45a8-907f-be4cb83f744e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:32.550990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8fcefdda-6e37-45a8-907f-be4cb83f744e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:32.580717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8fcefdda-6e37-45a8-907f-be4cb83f744e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043771 May 25 16:33:32.581195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 500/982] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:32 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:33.597307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9de73e59-3bcf-4361-a41c-065e4098540a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:33.598404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9de73e59-3bcf-4361-a41c-065e4098540a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:33.601758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=31 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:33.609600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9de73e59-3bcf-4361-a41c-065e4098540a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:33.610171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9de73e59-3bcf-4361-a41c-065e4098540a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:33.611239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9de73e59-3bcf-4361-a41c-065e4098540a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:33.611784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9de73e59-3bcf-4361-a41c-065e4098540a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:33.615922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:33.646008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9de73e59-3bcf-4361-a41c-065e4098540a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055059 May 25 16:33:33.646560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 483/983] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:33 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:33:34.663185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4bfa8b1b-48c3-4ff6-b4c4-857f0439bd12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:34.664690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4bfa8b1b-48c3-4ff6-b4c4-857f0439bd12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:34.668181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=31 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:34.673839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4bfa8b1b-48c3-4ff6-b4c4-857f0439bd12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:34.674306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4bfa8b1b-48c3-4ff6-b4c4-857f0439bd12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:34.675064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4bfa8b1b-48c3-4ff6-b4c4-857f0439bd12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:34.675534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4bfa8b1b-48c3-4ff6-b4c4-857f0439bd12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:34.677887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=91 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:34.713686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4bfa8b1b-48c3-4ff6-b4c4-857f0439bd12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057004 May 25 16:33:34.714169 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 501/984] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:34 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:35.729418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dfd793a3-e3fa-4f0b-ba99-619cec345d6d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:35.730936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dfd793a3-e3fa-4f0b-ba99-619cec345d6d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:35.742155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dfd793a3-e3fa-4f0b-ba99-619cec345d6d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:35.742915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dfd793a3-e3fa-4f0b-ba99-619cec345d6d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:35.744221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dfd793a3-e3fa-4f0b-ba99-619cec345d6d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:35.745073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dfd793a3-e3fa-4f0b-ba99-619cec345d6d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:35.775992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dfd793a3-e3fa-4f0b-ba99-619cec345d6d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053231 May 25 16:33:35.776581 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 484/985] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:35 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:36.793416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-513ccac6-3176-4a9a-a3e7-21591e6c485c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:36.794424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-513ccac6-3176-4a9a-a3e7-21591e6c485c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:36.801281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-513ccac6-3176-4a9a-a3e7-21591e6c485c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:36.801735 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-513ccac6-3176-4a9a-a3e7-21591e6c485c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:36.802497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-513ccac6-3176-4a9a-a3e7-21591e6c485c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:36.802933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-513ccac6-3176-4a9a-a3e7-21591e6c485c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:36.832266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-513ccac6-3176-4a9a-a3e7-21591e6c485c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045054 May 25 16:33:36.832748 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 502/986] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:36 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:37.848646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-075e6a74-1863-4e91-8b99-6325299b7200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:37.850260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-075e6a74-1863-4e91-8b99-6325299b7200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:37.861304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-075e6a74-1863-4e91-8b99-6325299b7200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:37.862084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-075e6a74-1863-4e91-8b99-6325299b7200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:37.863389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-075e6a74-1863-4e91-8b99-6325299b7200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:37.864135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-075e6a74-1863-4e91-8b99-6325299b7200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:37.896515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-075e6a74-1863-4e91-8b99-6325299b7200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054586 May 25 16:33:37.897068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 485/987] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:37 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:38.912636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-316641e9-c271-440f-b4f0-adefbdade6e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:38.913982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-316641e9-c271-440f-b4f0-adefbdade6e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:38.922357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-316641e9-c271-440f-b4f0-adefbdade6e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:38.922816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-316641e9-c271-440f-b4f0-adefbdade6e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:38.923628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-316641e9-c271-440f-b4f0-adefbdade6e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:38.924068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-316641e9-c271-440f-b4f0-adefbdade6e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:38.957454 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-316641e9-c271-440f-b4f0-adefbdade6e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051697 May 25 16:33:38.958024 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 503/988] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:38 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:39.973351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d2529170-90cc-44f6-a395-776c831af7c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:39.974718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d2529170-90cc-44f6-a395-776c831af7c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:39.983498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2529170-90cc-44f6-a395-776c831af7c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:39.983954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2529170-90cc-44f6-a395-776c831af7c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:39.984726 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2529170-90cc-44f6-a395-776c831af7c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:39.985181 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2529170-90cc-44f6-a395-776c831af7c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:40.015937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d2529170-90cc-44f6-a395-776c831af7c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049297 May 25 16:33:40.016491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 486/989] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:39 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:41.038027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6dceb581-4c3b-4c4d-893e-2a3262dd9f27 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:41.039549 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6dceb581-4c3b-4c4d-893e-2a3262dd9f27 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:41.054676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6dceb581-4c3b-4c4d-893e-2a3262dd9f27 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:41.055466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6dceb581-4c3b-4c4d-893e-2a3262dd9f27 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:41.056722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6dceb581-4c3b-4c4d-893e-2a3262dd9f27 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:41.057500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6dceb581-4c3b-4c4d-893e-2a3262dd9f27 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:41.089307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6dceb581-4c3b-4c4d-893e-2a3262dd9f27 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064107 May 25 16:33:41.089866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 504/990] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:41 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:42.104013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3d213c19-1103-4418-b8f7-558e62bb7083 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:42.105552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3d213c19-1103-4418-b8f7-558e62bb7083 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:42.112423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d213c19-1103-4418-b8f7-558e62bb7083 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:42.112818 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d213c19-1103-4418-b8f7-558e62bb7083 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:42.113577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d213c19-1103-4418-b8f7-558e62bb7083 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:42.114018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d213c19-1103-4418-b8f7-558e62bb7083 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:42.148764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3d213c19-1103-4418-b8f7-558e62bb7083 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050399 May 25 16:33:42.149555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 487/991] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:42 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:43.166316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2a4d0e0f-0575-490d-8237-293c30970157 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:43.167829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2a4d0e0f-0575-490d-8237-293c30970157 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:43.178122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a4d0e0f-0575-490d-8237-293c30970157 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:43.178786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a4d0e0f-0575-490d-8237-293c30970157 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:43.179958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a4d0e0f-0575-490d-8237-293c30970157 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:43.180450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a4d0e0f-0575-490d-8237-293c30970157 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:43.210793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2a4d0e0f-0575-490d-8237-293c30970157 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051204 May 25 16:33:43.211294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 505/992] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:43 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:44.226582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0a39938b-1489-437d-860c-9bdb58b812ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:44.228108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0a39938b-1489-437d-860c-9bdb58b812ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:44.235509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a39938b-1489-437d-860c-9bdb58b812ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:44.236035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a39938b-1489-437d-860c-9bdb58b812ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:44.236977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a39938b-1489-437d-860c-9bdb58b812ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:44.237420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a39938b-1489-437d-860c-9bdb58b812ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:44.267763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0a39938b-1489-437d-860c-9bdb58b812ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047826 May 25 16:33:44.268310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 488/993] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:44 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:45.283814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a0ef4b3e-11ed-4c6f-9492-3484bb65c268 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:45.285316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a0ef4b3e-11ed-4c6f-9492-3484bb65c268 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:45.294891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a0ef4b3e-11ed-4c6f-9492-3484bb65c268 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:45.295386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a0ef4b3e-11ed-4c6f-9492-3484bb65c268 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:45.296157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a0ef4b3e-11ed-4c6f-9492-3484bb65c268 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:45.296597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a0ef4b3e-11ed-4c6f-9492-3484bb65c268 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:45.326454 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a0ef4b3e-11ed-4c6f-9492-3484bb65c268 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049400 May 25 16:33:45.327011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 506/994] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:45 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:46.341911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bcd5ac24-c543-4223-9cee-89f822bffba3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:46.343192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bcd5ac24-c543-4223-9cee-89f822bffba3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:46.352506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bcd5ac24-c543-4223-9cee-89f822bffba3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:46.353117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bcd5ac24-c543-4223-9cee-89f822bffba3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:46.354178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bcd5ac24-c543-4223-9cee-89f822bffba3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:46.354768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bcd5ac24-c543-4223-9cee-89f822bffba3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:46.388008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bcd5ac24-c543-4223-9cee-89f822bffba3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052106 May 25 16:33:46.388593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 489/995] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:46 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:47.403697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d0fbf2dc-9e9b-4a39-9bf8-ef9d33e939a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:47.405246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d0fbf2dc-9e9b-4a39-9bf8-ef9d33e939a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:47.416742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0fbf2dc-9e9b-4a39-9bf8-ef9d33e939a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:47.417189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0fbf2dc-9e9b-4a39-9bf8-ef9d33e939a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:47.417981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0fbf2dc-9e9b-4a39-9bf8-ef9d33e939a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:47.418412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0fbf2dc-9e9b-4a39-9bf8-ef9d33e939a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:47.447964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d0fbf2dc-9e9b-4a39-9bf8-ef9d33e939a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050887 May 25 16:33:47.448522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 507/996] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:47 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:48.463783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-55615a88-9440-444a-aced-b3d6397d933a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:48.464782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-55615a88-9440-444a-aced-b3d6397d933a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:48.472163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55615a88-9440-444a-aced-b3d6397d933a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:48.472644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55615a88-9440-444a-aced-b3d6397d933a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:48.473505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55615a88-9440-444a-aced-b3d6397d933a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:48.473988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55615a88-9440-444a-aced-b3d6397d933a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:48.505312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-55615a88-9440-444a-aced-b3d6397d933a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048549 May 25 16:33:48.505879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 490/997] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:48 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:49.522020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-90ada77d-faec-41f5-a444-1669febc8b95 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:49.523444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-90ada77d-faec-41f5-a444-1669febc8b95 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:49.533821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90ada77d-faec-41f5-a444-1669febc8b95 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:49.534618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90ada77d-faec-41f5-a444-1669febc8b95 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:49.535943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90ada77d-faec-41f5-a444-1669febc8b95 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:49.536666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90ada77d-faec-41f5-a444-1669febc8b95 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:49.568458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-90ada77d-faec-41f5-a444-1669febc8b95 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053657 May 25 16:33:49.569017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 508/998] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:49 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:50.584613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9654322e-e852-45a0-b270-82fc459931a6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:50.586159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9654322e-e852-45a0-b270-82fc459931a6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:50.593899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9654322e-e852-45a0-b270-82fc459931a6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:50.594354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9654322e-e852-45a0-b270-82fc459931a6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:50.595120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9654322e-e852-45a0-b270-82fc459931a6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:50.595593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9654322e-e852-45a0-b270-82fc459931a6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:50.624905 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9654322e-e852-45a0-b270-82fc459931a6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047036 May 25 16:33:50.625442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 491/999] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:50 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:51.640991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-abfcefe3-1c73-40dc-a730-9c846c1c9768 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:51.642410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-abfcefe3-1c73-40dc-a730-9c846c1c9768 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:51.652405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-abfcefe3-1c73-40dc-a730-9c846c1c9768 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:51.652861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-abfcefe3-1c73-40dc-a730-9c846c1c9768 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:51.653627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-abfcefe3-1c73-40dc-a730-9c846c1c9768 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:51.654057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-abfcefe3-1c73-40dc-a730-9c846c1c9768 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:51.684654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-abfcefe3-1c73-40dc-a730-9c846c1c9768 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050539 May 25 16:33:51.685209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 509/1000] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:51 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:52.696573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bc7f10fd-6289-4e58-a53c-b15ebb0a33fe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:52.697543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bc7f10fd-6289-4e58-a53c-b15ebb0a33fe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:52.704410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc7f10fd-6289-4e58-a53c-b15ebb0a33fe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:52.704858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc7f10fd-6289-4e58-a53c-b15ebb0a33fe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:52.705620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc7f10fd-6289-4e58-a53c-b15ebb0a33fe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:52.706059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc7f10fd-6289-4e58-a53c-b15ebb0a33fe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:52.736566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bc7f10fd-6289-4e58-a53c-b15ebb0a33fe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044032 May 25 16:33:52.737122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 492/1001] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:52 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:53.748587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1ca42340-06ac-4eb7-a34a-8ae724dd9be3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:53.750426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1ca42340-06ac-4eb7-a34a-8ae724dd9be3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:53.757199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ca42340-06ac-4eb7-a34a-8ae724dd9be3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:53.757659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ca42340-06ac-4eb7-a34a-8ae724dd9be3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:53.758425 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ca42340-06ac-4eb7-a34a-8ae724dd9be3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:53.758867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ca42340-06ac-4eb7-a34a-8ae724dd9be3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:53.789220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1ca42340-06ac-4eb7-a34a-8ae724dd9be3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044688 May 25 16:33:53.789983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 510/1002] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:53 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:54.808491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8bf4fb00-6448-4006-a272-4e1e51946d79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:54.810003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8bf4fb00-6448-4006-a272-4e1e51946d79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:54.821596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bf4fb00-6448-4006-a272-4e1e51946d79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:54.822353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bf4fb00-6448-4006-a272-4e1e51946d79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:54.823323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bf4fb00-6448-4006-a272-4e1e51946d79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:54.823761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bf4fb00-6448-4006-a272-4e1e51946d79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:54.853875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8bf4fb00-6448-4006-a272-4e1e51946d79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052181 May 25 16:33:54.854415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 493/1003] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:54 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:55.869812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3ba038a3-fb83-452d-8e06-47793e04cb2c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:55.870956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3ba038a3-fb83-452d-8e06-47793e04cb2c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:55.879530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ba038a3-fb83-452d-8e06-47793e04cb2c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:55.880520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ba038a3-fb83-452d-8e06-47793e04cb2c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:55.881811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ba038a3-fb83-452d-8e06-47793e04cb2c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:55.882349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ba038a3-fb83-452d-8e06-47793e04cb2c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:55.918610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3ba038a3-fb83-452d-8e06-47793e04cb2c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055461 May 25 16:33:55.919190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 511/1004] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:55 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:56.934165 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-95e1746b-66b9-4b16-bcd8-1ab339755990 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:56.935558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-95e1746b-66b9-4b16-bcd8-1ab339755990 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:56.946750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-95e1746b-66b9-4b16-bcd8-1ab339755990 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:56.947707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-95e1746b-66b9-4b16-bcd8-1ab339755990 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:56.948481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-95e1746b-66b9-4b16-bcd8-1ab339755990 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:56.948921 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-95e1746b-66b9-4b16-bcd8-1ab339755990 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:56.979294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-95e1746b-66b9-4b16-bcd8-1ab339755990 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051970 May 25 16:33:56.979860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 494/1005] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:56 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:57.992856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-209d5811-f633-4379-b82c-82952120be24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:57.993695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-209d5811-f633-4379-b82c-82952120be24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:58.000539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-209d5811-f633-4379-b82c-82952120be24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:58.000995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-209d5811-f633-4379-b82c-82952120be24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:58.001762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-209d5811-f633-4379-b82c-82952120be24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:58.002198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-209d5811-f633-4379-b82c-82952120be24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:58.032604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-209d5811-f633-4379-b82c-82952120be24 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043903 May 25 16:33:58.033166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 512/1006] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:57 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:33:59.049401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5e79baaa-1f6b-4282-9bbd-5f9d5db9c0e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:33:59.051000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5e79baaa-1f6b-4282-9bbd-5f9d5db9c0e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:33:59.059785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e79baaa-1f6b-4282-9bbd-5f9d5db9c0e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:59.060245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e79baaa-1f6b-4282-9bbd-5f9d5db9c0e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:59.061020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e79baaa-1f6b-4282-9bbd-5f9d5db9c0e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:33:59.061458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e79baaa-1f6b-4282-9bbd-5f9d5db9c0e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:33:59.092021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5e79baaa-1f6b-4282-9bbd-5f9d5db9c0e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049583 May 25 16:33:59.092583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 495/1007] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:33:59 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:00.106466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b08e1038-e0e4-476a-811a-d5827b7a45e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:00.107430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b08e1038-e0e4-476a-811a-d5827b7a45e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:00.114394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b08e1038-e0e4-476a-811a-d5827b7a45e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:00.114843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b08e1038-e0e4-476a-811a-d5827b7a45e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:00.115654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b08e1038-e0e4-476a-811a-d5827b7a45e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:00.116067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b08e1038-e0e4-476a-811a-d5827b7a45e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:00.146012 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b08e1038-e0e4-476a-811a-d5827b7a45e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044446 May 25 16:34:00.146674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 513/1008] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:00 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:01.162199 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c89f0f6c-a601-42ef-ad45-84c2f969646e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:01.163625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c89f0f6c-a601-42ef-ad45-84c2f969646e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:01.174942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c89f0f6c-a601-42ef-ad45-84c2f969646e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:01.175766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c89f0f6c-a601-42ef-ad45-84c2f969646e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:01.177025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c89f0f6c-a601-42ef-ad45-84c2f969646e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:01.177752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c89f0f6c-a601-42ef-ad45-84c2f969646e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:01.209647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c89f0f6c-a601-42ef-ad45-84c2f969646e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054195 May 25 16:34:01.210210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 496/1009] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:01 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:02.225820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b9272a3d-6cf3-48e0-9849-7c926598f094 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:02.227225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b9272a3d-6cf3-48e0-9849-7c926598f094 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:02.234605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9272a3d-6cf3-48e0-9849-7c926598f094 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:02.235038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9272a3d-6cf3-48e0-9849-7c926598f094 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:02.235838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9272a3d-6cf3-48e0-9849-7c926598f094 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:02.236277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9272a3d-6cf3-48e0-9849-7c926598f094 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:02.265951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b9272a3d-6cf3-48e0-9849-7c926598f094 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047005 May 25 16:34:02.266513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 514/1010] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:02 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:03.281769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c73145e4-2922-4887-93ad-a732e9dbfedb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:03.283475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c73145e4-2922-4887-93ad-a732e9dbfedb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:03.293646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c73145e4-2922-4887-93ad-a732e9dbfedb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:03.294098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c73145e4-2922-4887-93ad-a732e9dbfedb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:03.294855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c73145e4-2922-4887-93ad-a732e9dbfedb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:03.295312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c73145e4-2922-4887-93ad-a732e9dbfedb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:03.307765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:34:03.308244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:34:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:34:03.325937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c73145e4-2922-4887-93ad-a732e9dbfedb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050803 May 25 16:34:03.326502 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 497/1011] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:03 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:04.341783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-01839532-5e71-4097-8b57-11407547081d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:04.342648 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-01839532-5e71-4097-8b57-11407547081d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:04.349595 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01839532-5e71-4097-8b57-11407547081d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:04.350034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01839532-5e71-4097-8b57-11407547081d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:04.350821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01839532-5e71-4097-8b57-11407547081d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:04.351293 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01839532-5e71-4097-8b57-11407547081d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:04.381678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-01839532-5e71-4097-8b57-11407547081d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044719 May 25 16:34:04.382235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 515/1012] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:04 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:05.397454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e2534d67-2166-41d8-a0e3-34e3360c5206 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:05.398874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e2534d67-2166-41d8-a0e3-34e3360c5206 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:05.402593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:05.411698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2534d67-2166-41d8-a0e3-34e3360c5206 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:05.412449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2534d67-2166-41d8-a0e3-34e3360c5206 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:05.413704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2534d67-2166-41d8-a0e3-34e3360c5206 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:05.414447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2534d67-2166-41d8-a0e3-34e3360c5206 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:05.417975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=90 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:05.445796 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e2534d67-2166-41d8-a0e3-34e3360c5206 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055156 May 25 16:34:05.446470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 498/1013] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:05 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:06.461572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fb6daaae-458b-4d8b-9860-c4b8d44b8b71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:06.463002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fb6daaae-458b-4d8b-9860-c4b8d44b8b71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:06.466168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=30 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:06.475080 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb6daaae-458b-4d8b-9860-c4b8d44b8b71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:06.475896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb6daaae-458b-4d8b-9860-c4b8d44b8b71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:06.477100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb6daaae-458b-4d8b-9860-c4b8d44b8b71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:06.477889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb6daaae-458b-4d8b-9860-c4b8d44b8b71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:06.480668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=90 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:06.518977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fb6daaae-458b-4d8b-9860-c4b8d44b8b71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064193 May 25 16:34:06.519557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 516/1014] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:06 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:07.535225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ff1e3f16-2128-4478-a10d-9ada9f323425 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:07.536692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ff1e3f16-2128-4478-a10d-9ada9f323425 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:07.545455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff1e3f16-2128-4478-a10d-9ada9f323425 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:07.545896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff1e3f16-2128-4478-a10d-9ada9f323425 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:07.546657 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff1e3f16-2128-4478-a10d-9ada9f323425 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:07.547094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff1e3f16-2128-4478-a10d-9ada9f323425 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:07.579963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ff1e3f16-2128-4478-a10d-9ada9f323425 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051884 May 25 16:34:07.580521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 499/1015] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:07 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:08.594258 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bbd6c91a-d142-40fd-9a69-026d602f3d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:08.595080 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bbd6c91a-d142-40fd-9a69-026d602f3d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:08.602106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bbd6c91a-d142-40fd-9a69-026d602f3d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:08.602557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bbd6c91a-d142-40fd-9a69-026d602f3d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:08.603438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bbd6c91a-d142-40fd-9a69-026d602f3d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:08.603907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bbd6c91a-d142-40fd-9a69-026d602f3d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:08.635571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bbd6c91a-d142-40fd-9a69-026d602f3d4b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045691 May 25 16:34:08.636129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 517/1016] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:08 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:09.650802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-46013336-7fe6-4163-9c36-e55d5e6719d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:09.651909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-46013336-7fe6-4163-9c36-e55d5e6719d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:09.659871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46013336-7fe6-4163-9c36-e55d5e6719d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:09.660399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46013336-7fe6-4163-9c36-e55d5e6719d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:09.661305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46013336-7fe6-4163-9c36-e55d5e6719d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:09.661815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46013336-7fe6-4163-9c36-e55d5e6719d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:09.696545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-46013336-7fe6-4163-9c36-e55d5e6719d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050383 May 25 16:34:09.697363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 500/1017] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:09 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:10.712849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-edca1154-3723-47b1-ad97-be753f662b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:10.714264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-edca1154-3723-47b1-ad97-be753f662b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:10.725095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edca1154-3723-47b1-ad97-be753f662b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:10.725868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edca1154-3723-47b1-ad97-be753f662b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:10.727105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edca1154-3723-47b1-ad97-be753f662b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:10.727880 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edca1154-3723-47b1-ad97-be753f662b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:10.758315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-edca1154-3723-47b1-ad97-be753f662b40 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052266 May 25 16:34:10.758881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 518/1018] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:10 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:11.774529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a4e71aeb-7ec4-4f8d-9ed2-cd5fee2a8f3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:11.775829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a4e71aeb-7ec4-4f8d-9ed2-cd5fee2a8f3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:11.782907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a4e71aeb-7ec4-4f8d-9ed2-cd5fee2a8f3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:11.783397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a4e71aeb-7ec4-4f8d-9ed2-cd5fee2a8f3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:11.784164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a4e71aeb-7ec4-4f8d-9ed2-cd5fee2a8f3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:11.784605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a4e71aeb-7ec4-4f8d-9ed2-cd5fee2a8f3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:11.814430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a4e71aeb-7ec4-4f8d-9ed2-cd5fee2a8f3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046607 May 25 16:34:11.814977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 501/1019] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:11 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:12.830390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-45e11058-0553-43c3-b278-e482f6d73ad5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:12.831823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-45e11058-0553-43c3-b278-e482f6d73ad5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:12.842846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45e11058-0553-43c3-b278-e482f6d73ad5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:12.843628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45e11058-0553-43c3-b278-e482f6d73ad5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:12.844888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45e11058-0553-43c3-b278-e482f6d73ad5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:12.845614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45e11058-0553-43c3-b278-e482f6d73ad5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:12.878333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-45e11058-0553-43c3-b278-e482f6d73ad5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054724 May 25 16:34:12.878870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 519/1020] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:12 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:13.894424 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-06c9fbc5-9de6-48a1-8a2b-15a5f9f9c58e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:13.895808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-06c9fbc5-9de6-48a1-8a2b-15a5f9f9c58e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:13.904406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-06c9fbc5-9de6-48a1-8a2b-15a5f9f9c58e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:13.904862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-06c9fbc5-9de6-48a1-8a2b-15a5f9f9c58e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:13.905630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-06c9fbc5-9de6-48a1-8a2b-15a5f9f9c58e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:13.906072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-06c9fbc5-9de6-48a1-8a2b-15a5f9f9c58e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:13.938537 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-06c9fbc5-9de6-48a1-8a2b-15a5f9f9c58e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050942 May 25 16:34:13.939093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 502/1021] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:13 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:14.953684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-de62b472-404c-414f-be86-8b5d4e3fd4e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:14.955111 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-de62b472-404c-414f-be86-8b5d4e3fd4e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:14.966167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-de62b472-404c-414f-be86-8b5d4e3fd4e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:14.966917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-de62b472-404c-414f-be86-8b5d4e3fd4e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:14.968234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-de62b472-404c-414f-be86-8b5d4e3fd4e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:14.969069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-de62b472-404c-414f-be86-8b5d4e3fd4e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:15.002062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-de62b472-404c-414f-be86-8b5d4e3fd4e0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055302 May 25 16:34:15.002591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 520/1022] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:14 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:16.015087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7ba5cb40-e88f-4754-bf8f-2d233b07c631 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:16.016001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7ba5cb40-e88f-4754-bf8f-2d233b07c631 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:16.022896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ba5cb40-e88f-4754-bf8f-2d233b07c631 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:16.023394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ba5cb40-e88f-4754-bf8f-2d233b07c631 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:16.024165 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ba5cb40-e88f-4754-bf8f-2d233b07c631 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:16.024736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ba5cb40-e88f-4754-bf8f-2d233b07c631 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:16.054663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7ba5cb40-e88f-4754-bf8f-2d233b07c631 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043665 May 25 16:34:16.055245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 503/1023] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:16 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:17.071234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ae7bb2b5-b5e3-42b9-98db-a6bf7b73f929 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:17.073773 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ae7bb2b5-b5e3-42b9-98db-a6bf7b73f929 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:17.083464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ae7bb2b5-b5e3-42b9-98db-a6bf7b73f929 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:17.084220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ae7bb2b5-b5e3-42b9-98db-a6bf7b73f929 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:17.085522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ae7bb2b5-b5e3-42b9-98db-a6bf7b73f929 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:17.086260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ae7bb2b5-b5e3-42b9-98db-a6bf7b73f929 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:17.116848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ae7bb2b5-b5e3-42b9-98db-a6bf7b73f929 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052373 May 25 16:34:17.117414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 521/1024] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:17 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:18.129077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b7b8f8dc-d6d9-434c-817d-0842e0795c8d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:18.129986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b7b8f8dc-d6d9-434c-817d-0842e0795c8d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:18.137118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7b8f8dc-d6d9-434c-817d-0842e0795c8d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:18.137575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7b8f8dc-d6d9-434c-817d-0842e0795c8d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:18.138337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7b8f8dc-d6d9-434c-817d-0842e0795c8d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:18.138779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7b8f8dc-d6d9-434c-817d-0842e0795c8d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:18.169210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b7b8f8dc-d6d9-434c-817d-0842e0795c8d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044224 May 25 16:34:18.169780 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 504/1025] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:18 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:18.773217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:34:18.773377 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:34:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:34:19.181674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-047a39a0-db19-46fc-ad4a-30e70e303302 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:19.182588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-047a39a0-db19-46fc-ad4a-30e70e303302 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:19.189190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-047a39a0-db19-46fc-ad4a-30e70e303302 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:19.189589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-047a39a0-db19-46fc-ad4a-30e70e303302 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:19.190351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-047a39a0-db19-46fc-ad4a-30e70e303302 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:19.190785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-047a39a0-db19-46fc-ad4a-30e70e303302 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:19.220235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-047a39a0-db19-46fc-ad4a-30e70e303302 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042650 May 25 16:34:19.220781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 522/1026] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:19 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:20.232788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ccbc77dc-5a44-4f57-99ab-e78452561453 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:20.233713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ccbc77dc-5a44-4f57-99ab-e78452561453 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:20.240699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccbc77dc-5a44-4f57-99ab-e78452561453 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:20.241153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccbc77dc-5a44-4f57-99ab-e78452561453 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:20.241926 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccbc77dc-5a44-4f57-99ab-e78452561453 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:20.242360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccbc77dc-5a44-4f57-99ab-e78452561453 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:20.272967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ccbc77dc-5a44-4f57-99ab-e78452561453 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044122 May 25 16:34:20.273477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 505/1027] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:20 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:21.286055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-758b68b9-7dd8-4d13-91d7-ee8145f69135 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:21.286992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-758b68b9-7dd8-4d13-91d7-ee8145f69135 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:21.293999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-758b68b9-7dd8-4d13-91d7-ee8145f69135 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:21.294451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-758b68b9-7dd8-4d13-91d7-ee8145f69135 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:21.295232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-758b68b9-7dd8-4d13-91d7-ee8145f69135 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:21.295667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-758b68b9-7dd8-4d13-91d7-ee8145f69135 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:21.324225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-758b68b9-7dd8-4d13-91d7-ee8145f69135 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042042 May 25 16:34:21.324705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 523/1028] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:21 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:22.336307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1a3cf5aa-b308-4f6a-a267-3296bb246e82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:22.337200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1a3cf5aa-b308-4f6a-a267-3296bb246e82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:22.343918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a3cf5aa-b308-4f6a-a267-3296bb246e82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:22.344371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a3cf5aa-b308-4f6a-a267-3296bb246e82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:22.345137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a3cf5aa-b308-4f6a-a267-3296bb246e82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:22.345573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a3cf5aa-b308-4f6a-a267-3296bb246e82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:22.375247 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1a3cf5aa-b308-4f6a-a267-3296bb246e82 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042924 May 25 16:34:22.375803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 506/1029] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:22 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:23.386167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2603c07b-bd88-441d-8b8e-3909f127070b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:23.387389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2603c07b-bd88-441d-8b8e-3909f127070b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:23.394166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2603c07b-bd88-441d-8b8e-3909f127070b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:23.394840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2603c07b-bd88-441d-8b8e-3909f127070b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:23.395931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2603c07b-bd88-441d-8b8e-3909f127070b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:23.396388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2603c07b-bd88-441d-8b8e-3909f127070b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:23.428155 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2603c07b-bd88-441d-8b8e-3909f127070b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046007 May 25 16:34:23.432154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 524/1030] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:23 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:24.439960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-39ab470a-8a9b-414b-bb27-94fba6927304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:24.440913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-39ab470a-8a9b-414b-bb27-94fba6927304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:24.447782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39ab470a-8a9b-414b-bb27-94fba6927304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:24.448237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39ab470a-8a9b-414b-bb27-94fba6927304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:24.449096 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39ab470a-8a9b-414b-bb27-94fba6927304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:24.449543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39ab470a-8a9b-414b-bb27-94fba6927304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:24.479509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-39ab470a-8a9b-414b-bb27-94fba6927304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043421 May 25 16:34:24.480065 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 507/1031] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:24 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:25.491958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-88e680b2-f02f-4d2e-a127-d00e5b74668b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:25.493404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-88e680b2-f02f-4d2e-a127-d00e5b74668b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:25.499841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88e680b2-f02f-4d2e-a127-d00e5b74668b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:25.500287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88e680b2-f02f-4d2e-a127-d00e5b74668b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:25.501091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88e680b2-f02f-4d2e-a127-d00e5b74668b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:25.501517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88e680b2-f02f-4d2e-a127-d00e5b74668b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:25.532731 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-88e680b2-f02f-4d2e-a127-d00e5b74668b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045424 May 25 16:34:25.533288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 525/1032] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:25 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:26.544831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-33823df2-e88d-4a5c-a45d-98077aa59d0e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:26.545801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-33823df2-e88d-4a5c-a45d-98077aa59d0e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:26.552809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-33823df2-e88d-4a5c-a45d-98077aa59d0e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:26.553325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-33823df2-e88d-4a5c-a45d-98077aa59d0e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:26.554104 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-33823df2-e88d-4a5c-a45d-98077aa59d0e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:26.554533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-33823df2-e88d-4a5c-a45d-98077aa59d0e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:26.584992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-33823df2-e88d-4a5c-a45d-98077aa59d0e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044537 May 25 16:34:26.588840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 508/1033] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:26 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:27.595959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6ec6527c-1ecf-4da1-b43d-9a94d7bfa120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:27.597533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6ec6527c-1ecf-4da1-b43d-9a94d7bfa120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:27.604373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ec6527c-1ecf-4da1-b43d-9a94d7bfa120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:27.604823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ec6527c-1ecf-4da1-b43d-9a94d7bfa120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:27.605581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ec6527c-1ecf-4da1-b43d-9a94d7bfa120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:27.606015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ec6527c-1ecf-4da1-b43d-9a94d7bfa120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:27.637077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6ec6527c-1ecf-4da1-b43d-9a94d7bfa120 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045220 May 25 16:34:27.637622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 526/1034] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:27 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:28.649068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-15748ea5-4ed5-4aa7-ae40-d76f5997430a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:28.650055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-15748ea5-4ed5-4aa7-ae40-d76f5997430a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:28.656828 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-15748ea5-4ed5-4aa7-ae40-d76f5997430a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:28.657284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-15748ea5-4ed5-4aa7-ae40-d76f5997430a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:28.658050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-15748ea5-4ed5-4aa7-ae40-d76f5997430a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:28.658489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-15748ea5-4ed5-4aa7-ae40-d76f5997430a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:28.688964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-15748ea5-4ed5-4aa7-ae40-d76f5997430a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043996 May 25 16:34:28.689532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 509/1035] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:28 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:29.700991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-87234c7c-ac45-42fe-a973-b6691526487d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:29.702014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-87234c7c-ac45-42fe-a973-b6691526487d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:29.708791 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87234c7c-ac45-42fe-a973-b6691526487d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:29.709243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87234c7c-ac45-42fe-a973-b6691526487d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:29.710005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87234c7c-ac45-42fe-a973-b6691526487d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:29.710440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87234c7c-ac45-42fe-a973-b6691526487d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:29.744209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-87234c7c-ac45-42fe-a973-b6691526487d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046993 May 25 16:34:29.747751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 527/1036] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:29 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:30.755338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4eca9cde-f775-41e6-ad2f-899c2264a7ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:30.756459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4eca9cde-f775-41e6-ad2f-899c2264a7ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:30.763609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4eca9cde-f775-41e6-ad2f-899c2264a7ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:30.764137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4eca9cde-f775-41e6-ad2f-899c2264a7ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:30.765007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4eca9cde-f775-41e6-ad2f-899c2264a7ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:30.765514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4eca9cde-f775-41e6-ad2f-899c2264a7ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:30.795241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4eca9cde-f775-41e6-ad2f-899c2264a7ff tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043783 May 25 16:34:30.796093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 510/1037] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:30 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:31.806819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-952d83a3-068a-4c74-95ba-126debaf93a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:31.807822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-952d83a3-068a-4c74-95ba-126debaf93a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:31.814525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-952d83a3-068a-4c74-95ba-126debaf93a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:31.814980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-952d83a3-068a-4c74-95ba-126debaf93a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:31.815784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-952d83a3-068a-4c74-95ba-126debaf93a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:31.816225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-952d83a3-068a-4c74-95ba-126debaf93a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:31.846281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-952d83a3-068a-4c74-95ba-126debaf93a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043313 May 25 16:34:31.847067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 528/1038] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:31 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:32.857900 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-19dc21bc-a017-4a1e-a620-0cd321b5311d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:32.858945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-19dc21bc-a017-4a1e-a620-0cd321b5311d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:32.866185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19dc21bc-a017-4a1e-a620-0cd321b5311d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:32.866717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19dc21bc-a017-4a1e-a620-0cd321b5311d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:32.867607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19dc21bc-a017-4a1e-a620-0cd321b5311d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:32.868116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19dc21bc-a017-4a1e-a620-0cd321b5311d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:32.898974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-19dc21bc-a017-4a1e-a620-0cd321b5311d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045315 May 25 16:34:32.899566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 511/1039] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:32 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:33.910524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-179128e1-24e3-4b22-80d9-bc858c752d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:33.911481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-179128e1-24e3-4b22-80d9-bc858c752d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:33.918902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-179128e1-24e3-4b22-80d9-bc858c752d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:33.919481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-179128e1-24e3-4b22-80d9-bc858c752d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:33.920171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-179128e1-24e3-4b22-80d9-bc858c752d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:33.920607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-179128e1-24e3-4b22-80d9-bc858c752d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:33.951751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-179128e1-24e3-4b22-80d9-bc858c752d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045170 May 25 16:34:33.952323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 529/1040] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:33 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:34.962799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cb7df943-a535-4478-9d99-c2a79be9cee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:34.963724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cb7df943-a535-4478-9d99-c2a79be9cee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:34.970823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb7df943-a535-4478-9d99-c2a79be9cee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:34.971362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb7df943-a535-4478-9d99-c2a79be9cee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:34.972116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb7df943-a535-4478-9d99-c2a79be9cee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:34.972546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb7df943-a535-4478-9d99-c2a79be9cee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:35.004601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cb7df943-a535-4478-9d99-c2a79be9cee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045761 May 25 16:34:35.005167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 512/1041] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:34 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:36.016176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a66355f1-b0d5-411d-b06a-b61886e30d5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:36.017171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a66355f1-b0d5-411d-b06a-b61886e30d5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:36.023839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a66355f1-b0d5-411d-b06a-b61886e30d5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:36.024306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a66355f1-b0d5-411d-b06a-b61886e30d5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:36.025141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a66355f1-b0d5-411d-b06a-b61886e30d5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:36.025580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a66355f1-b0d5-411d-b06a-b61886e30d5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:36.055565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a66355f1-b0d5-411d-b06a-b61886e30d5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043464 May 25 16:34:36.056247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 530/1042] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:36 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:37.067606 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a7d35b2b-d723-4655-87ee-14dc22edd96a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:37.068539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a7d35b2b-d723-4655-87ee-14dc22edd96a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:37.070850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:37.076458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a7d35b2b-d723-4655-87ee-14dc22edd96a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:37.076897 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a7d35b2b-d723-4655-87ee-14dc22edd96a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:37.077874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a7d35b2b-d723-4655-87ee-14dc22edd96a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:37.078503 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a7d35b2b-d723-4655-87ee-14dc22edd96a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:37.081471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=90 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:37.112453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a7d35b2b-d723-4655-87ee-14dc22edd96a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048755 May 25 16:34:37.113304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 513/1043] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:37 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:38.124195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-38446552-54c5-41d9-8add-5448c33e0ac1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:38.125125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-38446552-54c5-41d9-8add-5448c33e0ac1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:38.128540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=30 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:38.132145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38446552-54c5-41d9-8add-5448c33e0ac1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:38.132590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38446552-54c5-41d9-8add-5448c33e0ac1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:38.133353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38446552-54c5-41d9-8add-5448c33e0ac1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:38.133793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38446552-54c5-41d9-8add-5448c33e0ac1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:38.137013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=90 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:38.165886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-38446552-54c5-41d9-8add-5448c33e0ac1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045339 May 25 16:34:38.166464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 531/1044] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:38 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:39.181553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-25dc9790-260c-45c1-abb8-f95149983b44 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:39.182416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-25dc9790-260c-45c1-abb8-f95149983b44 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:39.189397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25dc9790-260c-45c1-abb8-f95149983b44 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:39.189857 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25dc9790-260c-45c1-abb8-f95149983b44 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:39.190623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25dc9790-260c-45c1-abb8-f95149983b44 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:39.191060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25dc9790-260c-45c1-abb8-f95149983b44 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:39.221324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-25dc9790-260c-45c1-abb8-f95149983b44 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046248 May 25 16:34:39.222053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 514/1045] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:39 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:40.236943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4907e997-501d-4030-b6ea-ed0f431091a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:40.238440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4907e997-501d-4030-b6ea-ed0f431091a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:40.252511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4907e997-501d-4030-b6ea-ed0f431091a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:40.253298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4907e997-501d-4030-b6ea-ed0f431091a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:40.254625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4907e997-501d-4030-b6ea-ed0f431091a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:40.255361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4907e997-501d-4030-b6ea-ed0f431091a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:40.288577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4907e997-501d-4030-b6ea-ed0f431091a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058081 May 25 16:34:40.289143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 532/1046] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:40 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:41.305270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4431693e-5a98-45c2-9057-8363cac581d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:41.306490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4431693e-5a98-45c2-9057-8363cac581d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:41.313789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4431693e-5a98-45c2-9057-8363cac581d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:41.314279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4431693e-5a98-45c2-9057-8363cac581d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:41.315116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4431693e-5a98-45c2-9057-8363cac581d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:41.315621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4431693e-5a98-45c2-9057-8363cac581d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:41.347129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4431693e-5a98-45c2-9057-8363cac581d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048709 May 25 16:34:41.347729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 515/1047] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:41 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:42.363535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a9fde0c4-5e23-4ad2-ba35-d93355bf9295 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:42.365207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a9fde0c4-5e23-4ad2-ba35-d93355bf9295 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:42.376330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9fde0c4-5e23-4ad2-ba35-d93355bf9295 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:42.377088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9fde0c4-5e23-4ad2-ba35-d93355bf9295 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:42.378337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9fde0c4-5e23-4ad2-ba35-d93355bf9295 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:42.379073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9fde0c4-5e23-4ad2-ba35-d93355bf9295 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:42.413121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a9fde0c4-5e23-4ad2-ba35-d93355bf9295 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056563 May 25 16:34:42.413806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 533/1048] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:42 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:43.428960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8c685bc1-62ea-44f3-b42d-fbba6fc96a01 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:43.429988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8c685bc1-62ea-44f3-b42d-fbba6fc96a01 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:43.437297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c685bc1-62ea-44f3-b42d-fbba6fc96a01 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:43.437754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c685bc1-62ea-44f3-b42d-fbba6fc96a01 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:43.438542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c685bc1-62ea-44f3-b42d-fbba6fc96a01 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:43.438976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c685bc1-62ea-44f3-b42d-fbba6fc96a01 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:43.469855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8c685bc1-62ea-44f3-b42d-fbba6fc96a01 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047387 May 25 16:34:43.470397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 516/1049] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:43 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:44.486483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-26707b55-d67d-4458-9988-2b35c393f3c4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:44.487976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-26707b55-d67d-4458-9988-2b35c393f3c4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:44.497034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26707b55-d67d-4458-9988-2b35c393f3c4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:44.497492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26707b55-d67d-4458-9988-2b35c393f3c4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:44.498256 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26707b55-d67d-4458-9988-2b35c393f3c4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:44.498692 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26707b55-d67d-4458-9988-2b35c393f3c4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:44.528931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-26707b55-d67d-4458-9988-2b35c393f3c4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049241 May 25 16:34:44.529524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 534/1050] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:44 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:45.545459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-85467408-64c8-4d6d-95bf-20ba34d3ef52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:45.546788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-85467408-64c8-4d6d-95bf-20ba34d3ef52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:45.556545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85467408-64c8-4d6d-95bf-20ba34d3ef52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:45.557135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85467408-64c8-4d6d-95bf-20ba34d3ef52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:45.558097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85467408-64c8-4d6d-95bf-20ba34d3ef52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:45.558649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85467408-64c8-4d6d-95bf-20ba34d3ef52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:45.595376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-85467408-64c8-4d6d-95bf-20ba34d3ef52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056932 May 25 16:34:45.595865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 517/1051] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:45 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:46.613211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-19ea367e-f8d3-4789-9d6d-b311e66f771c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:46.614716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-19ea367e-f8d3-4789-9d6d-b311e66f771c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:46.623496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-19ea367e-f8d3-4789-9d6d-b311e66f771c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:46.624016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-19ea367e-f8d3-4789-9d6d-b311e66f771c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:46.624762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-19ea367e-f8d3-4789-9d6d-b311e66f771c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:46.625200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-19ea367e-f8d3-4789-9d6d-b311e66f771c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:46.656427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-19ea367e-f8d3-4789-9d6d-b311e66f771c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049953 May 25 16:34:46.657019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 535/1052] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:46 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:47.673225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6d0744ac-f2be-40d3-b853-15cfc4b1279d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:47.674694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6d0744ac-f2be-40d3-b853-15cfc4b1279d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:47.687348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d0744ac-f2be-40d3-b853-15cfc4b1279d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:47.688194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d0744ac-f2be-40d3-b853-15cfc4b1279d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:47.689441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d0744ac-f2be-40d3-b853-15cfc4b1279d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:47.690164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d0744ac-f2be-40d3-b853-15cfc4b1279d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:47.722557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6d0744ac-f2be-40d3-b853-15cfc4b1279d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056004 May 25 16:34:47.723472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 518/1053] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:47 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:48.738158 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-beb7b330-59af-4daa-8a0a-617e17900185 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:48.739676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-beb7b330-59af-4daa-8a0a-617e17900185 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:48.747665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-beb7b330-59af-4daa-8a0a-617e17900185 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:48.748114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-beb7b330-59af-4daa-8a0a-617e17900185 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:48.748872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-beb7b330-59af-4daa-8a0a-617e17900185 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:48.749304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-beb7b330-59af-4daa-8a0a-617e17900185 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:48.779566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-beb7b330-59af-4daa-8a0a-617e17900185 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048316 May 25 16:34:48.780358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 536/1054] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:48 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:49.795686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2826768f-ffdc-4297-b829-449a3f2ddba2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:49.797176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2826768f-ffdc-4297-b829-449a3f2ddba2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:49.807635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2826768f-ffdc-4297-b829-449a3f2ddba2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:49.808215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2826768f-ffdc-4297-b829-449a3f2ddba2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:49.809000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2826768f-ffdc-4297-b829-449a3f2ddba2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:49.809561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2826768f-ffdc-4297-b829-449a3f2ddba2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:49.840303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2826768f-ffdc-4297-b829-449a3f2ddba2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051653 May 25 16:34:49.841150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 519/1055] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:49 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:50.855650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cd79ca48-125e-4c04-9270-a029735b0d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:50.856804 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cd79ca48-125e-4c04-9270-a029735b0d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:50.865108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd79ca48-125e-4c04-9270-a029735b0d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:50.865669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd79ca48-125e-4c04-9270-a029735b0d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:50.866619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd79ca48-125e-4c04-9270-a029735b0d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:50.867269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd79ca48-125e-4c04-9270-a029735b0d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:50.901678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cd79ca48-125e-4c04-9270-a029735b0d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051746 May 25 16:34:50.902140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 537/1056] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:50 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:51.918025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b073a611-d0c1-4b1e-bcba-fc4d2cefd535 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:51.919596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b073a611-d0c1-4b1e-bcba-fc4d2cefd535 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:51.930948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b073a611-d0c1-4b1e-bcba-fc4d2cefd535 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:51.931229 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b073a611-d0c1-4b1e-bcba-fc4d2cefd535 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:51.932021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b073a611-d0c1-4b1e-bcba-fc4d2cefd535 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:51.932454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b073a611-d0c1-4b1e-bcba-fc4d2cefd535 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:51.962695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b073a611-d0c1-4b1e-bcba-fc4d2cefd535 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051695 May 25 16:34:51.963706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 520/1057] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:51 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:52.978257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8ec091a8-7afa-48c0-992c-b4d087b2e681 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:52.979264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8ec091a8-7afa-48c0-992c-b4d087b2e681 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:52.986775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ec091a8-7afa-48c0-992c-b4d087b2e681 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:52.987090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ec091a8-7afa-48c0-992c-b4d087b2e681 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:52.987976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ec091a8-7afa-48c0-992c-b4d087b2e681 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:52.988427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ec091a8-7afa-48c0-992c-b4d087b2e681 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:53.020847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8ec091a8-7afa-48c0-992c-b4d087b2e681 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048830 May 25 16:34:53.021502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 538/1058] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:52 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:54.036910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7ca331dd-32b9-46d8-9f82-b4a45f39f428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:54.038404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7ca331dd-32b9-46d8-9f82-b4a45f39f428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:54.050680 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca331dd-32b9-46d8-9f82-b4a45f39f428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:54.051498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca331dd-32b9-46d8-9f82-b4a45f39f428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:54.052801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca331dd-32b9-46d8-9f82-b4a45f39f428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:54.053525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ca331dd-32b9-46d8-9f82-b4a45f39f428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:54.085274 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7ca331dd-32b9-46d8-9f82-b4a45f39f428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055097 May 25 16:34:54.085860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 521/1059] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:54 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:55.099965 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ad32d28d-3eef-4c1e-ab1b-f4aa8a35ad63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:55.101100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ad32d28d-3eef-4c1e-ab1b-f4aa8a35ad63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:55.108379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad32d28d-3eef-4c1e-ab1b-f4aa8a35ad63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:55.108829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad32d28d-3eef-4c1e-ab1b-f4aa8a35ad63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:55.109588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad32d28d-3eef-4c1e-ab1b-f4aa8a35ad63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:55.110019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad32d28d-3eef-4c1e-ab1b-f4aa8a35ad63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:55.139950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ad32d28d-3eef-4c1e-ab1b-f4aa8a35ad63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046313 May 25 16:34:55.140521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 539/1060] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:55 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:56.156082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3917cdb7-97bf-4074-bb11-5945d5d59e17 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:56.157630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3917cdb7-97bf-4074-bb11-5945d5d59e17 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:56.167752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3917cdb7-97bf-4074-bb11-5945d5d59e17 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:56.168224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3917cdb7-97bf-4074-bb11-5945d5d59e17 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:56.168988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3917cdb7-97bf-4074-bb11-5945d5d59e17 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:56.169422 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3917cdb7-97bf-4074-bb11-5945d5d59e17 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:56.201203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3917cdb7-97bf-4074-bb11-5945d5d59e17 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051867 May 25 16:34:56.203394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 522/1061] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:56 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:57.215432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-50897e26-5fb8-4d78-ae22-7b104eb077eb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:57.216379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-50897e26-5fb8-4d78-ae22-7b104eb077eb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:57.223827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50897e26-5fb8-4d78-ae22-7b104eb077eb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:57.224095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50897e26-5fb8-4d78-ae22-7b104eb077eb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:57.224869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50897e26-5fb8-4d78-ae22-7b104eb077eb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:57.225300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-50897e26-5fb8-4d78-ae22-7b104eb077eb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:57.259520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-50897e26-5fb8-4d78-ae22-7b104eb077eb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048804 May 25 16:34:57.260081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 540/1062] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:57 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:58.275593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6b39bb8d-9452-4d60-91b5-181060ef087b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:58.277097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6b39bb8d-9452-4d60-91b5-181060ef087b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:58.287581 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b39bb8d-9452-4d60-91b5-181060ef087b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:58.288321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b39bb8d-9452-4d60-91b5-181060ef087b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:58.289556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b39bb8d-9452-4d60-91b5-181060ef087b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:58.290481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b39bb8d-9452-4d60-91b5-181060ef087b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:58.320562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6b39bb8d-9452-4d60-91b5-181060ef087b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051868 May 25 16:34:58.321194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 523/1063] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:58 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:34:59.336733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7f4218f2-16aa-4fba-81bd-4c0d6290dbca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:34:59.337651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7f4218f2-16aa-4fba-81bd-4c0d6290dbca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:34:59.346119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f4218f2-16aa-4fba-81bd-4c0d6290dbca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:59.346593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f4218f2-16aa-4fba-81bd-4c0d6290dbca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:59.347428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f4218f2-16aa-4fba-81bd-4c0d6290dbca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:34:59.347887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f4218f2-16aa-4fba-81bd-4c0d6290dbca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:34:59.379321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7f4218f2-16aa-4fba-81bd-4c0d6290dbca tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049417 May 25 16:34:59.380290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 541/1064] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:34:59 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:00.395379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ff89926c-c0c3-47b1-8fd4-c79340b77372 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:00.396775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ff89926c-c0c3-47b1-8fd4-c79340b77372 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:00.407693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff89926c-c0c3-47b1-8fd4-c79340b77372 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:00.408468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff89926c-c0c3-47b1-8fd4-c79340b77372 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:00.409722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff89926c-c0c3-47b1-8fd4-c79340b77372 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:00.410452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff89926c-c0c3-47b1-8fd4-c79340b77372 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:00.442475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ff89926c-c0c3-47b1-8fd4-c79340b77372 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053839 May 25 16:35:00.443042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 524/1065] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:00 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:01.458569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d18422dc-621d-4773-b8bf-e3ed781aa90e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:01.460121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d18422dc-621d-4773-b8bf-e3ed781aa90e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:01.468084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d18422dc-621d-4773-b8bf-e3ed781aa90e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:01.468543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d18422dc-621d-4773-b8bf-e3ed781aa90e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:01.469306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d18422dc-621d-4773-b8bf-e3ed781aa90e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:01.469737 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d18422dc-621d-4773-b8bf-e3ed781aa90e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:01.499761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d18422dc-621d-4773-b8bf-e3ed781aa90e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047821 May 25 16:35:01.500311 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 542/1066] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:01 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:02.515238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d5f8f2a8-d9f1-419d-bd92-d4a0782135fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:02.516810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d5f8f2a8-d9f1-419d-bd92-d4a0782135fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:02.526013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5f8f2a8-d9f1-419d-bd92-d4a0782135fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:02.526467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5f8f2a8-d9f1-419d-bd92-d4a0782135fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:02.527268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5f8f2a8-d9f1-419d-bd92-d4a0782135fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:02.527704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5f8f2a8-d9f1-419d-bd92-d4a0782135fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:02.558160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d5f8f2a8-d9f1-419d-bd92-d4a0782135fa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049670 May 25 16:35:02.558736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 525/1067] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:02 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:03.574231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b133b01d-054e-4690-81b3-48c4acbb8119 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:03.575759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b133b01d-054e-4690-81b3-48c4acbb8119 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:03.586737 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b133b01d-054e-4690-81b3-48c4acbb8119 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:03.587530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b133b01d-054e-4690-81b3-48c4acbb8119 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:03.588797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b133b01d-054e-4690-81b3-48c4acbb8119 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:03.589640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b133b01d-054e-4690-81b3-48c4acbb8119 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:03.624340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b133b01d-054e-4690-81b3-48c4acbb8119 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057009 May 25 16:35:03.624904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 543/1068] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:03 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:04.639832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-00774b13-ccef-4f9d-8a72-1bb4698e8b42 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:04.641373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-00774b13-ccef-4f9d-8a72-1bb4698e8b42 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:04.651175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00774b13-ccef-4f9d-8a72-1bb4698e8b42 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:04.651655 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00774b13-ccef-4f9d-8a72-1bb4698e8b42 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:04.652425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00774b13-ccef-4f9d-8a72-1bb4698e8b42 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:04.652863 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00774b13-ccef-4f9d-8a72-1bb4698e8b42 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:04.683434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-00774b13-ccef-4f9d-8a72-1bb4698e8b42 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050221 May 25 16:35:04.684226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 526/1069] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:04 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:05.698096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-23bd3d6e-f868-4643-8589-8b2b1fcd316a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:05.699136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-23bd3d6e-f868-4643-8589-8b2b1fcd316a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:05.706074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23bd3d6e-f868-4643-8589-8b2b1fcd316a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:05.706529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23bd3d6e-f868-4643-8589-8b2b1fcd316a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:05.707330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23bd3d6e-f868-4643-8589-8b2b1fcd316a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:05.707760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23bd3d6e-f868-4643-8589-8b2b1fcd316a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:05.737479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-23bd3d6e-f868-4643-8589-8b2b1fcd316a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044749 May 25 16:35:05.738049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 544/1070] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:05 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:06.754339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8962e4bc-d3fb-4c65-915e-071b5ae9e8cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:06.755886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8962e4bc-d3fb-4c65-915e-071b5ae9e8cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:06.766620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8962e4bc-d3fb-4c65-915e-071b5ae9e8cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:06.767428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8962e4bc-d3fb-4c65-915e-071b5ae9e8cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:06.768682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8962e4bc-d3fb-4c65-915e-071b5ae9e8cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:06.769413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8962e4bc-d3fb-4c65-915e-071b5ae9e8cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:06.804566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8962e4bc-d3fb-4c65-915e-071b5ae9e8cf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054560 May 25 16:35:06.804566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 527/1071] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:06 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:07.818646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a1749b4d-88e9-4583-a59f-f529f8045daa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:07.820211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a1749b4d-88e9-4583-a59f-f529f8045daa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:07.827457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1749b4d-88e9-4583-a59f-f529f8045daa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:07.828004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1749b4d-88e9-4583-a59f-f529f8045daa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:07.828674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1749b4d-88e9-4583-a59f-f529f8045daa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:07.829107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1749b4d-88e9-4583-a59f-f529f8045daa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:07.858474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a1749b4d-88e9-4583-a59f-f529f8045daa tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046535 May 25 16:35:07.859041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 545/1072] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:07 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:08.874812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6f2deef1-6984-4635-99cb-8afd89d2600a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:08.876336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6f2deef1-6984-4635-99cb-8afd89d2600a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:08.880027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:35:08.887820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6f2deef1-6984-4635-99cb-8afd89d2600a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:08.888301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6f2deef1-6984-4635-99cb-8afd89d2600a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:08.889069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6f2deef1-6984-4635-99cb-8afd89d2600a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:08.889532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6f2deef1-6984-4635-99cb-8afd89d2600a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:08.892588 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=90 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:35:08.921181 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6f2deef1-6984-4635-99cb-8afd89d2600a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053663 May 25 16:35:08.925033 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 528/1073] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:08 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:09.933160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0bfbee84-4226-4865-95d3-8b132546732e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:09.934109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0bfbee84-4226-4865-95d3-8b132546732e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:09.936514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=30 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:35:09.942323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bfbee84-4226-4865-95d3-8b132546732e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:09.942783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bfbee84-4226-4865-95d3-8b132546732e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:09.943636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bfbee84-4226-4865-95d3-8b132546732e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:09.944063 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bfbee84-4226-4865-95d3-8b132546732e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:09.948160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=90 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:35:09.975612 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0bfbee84-4226-4865-95d3-8b132546732e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046433 May 25 16:35:09.976290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 546/1074] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:09 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:10.987460 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ff3ff315-7fff-41eb-b9dd-2a4ab2286179 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:10.988399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ff3ff315-7fff-41eb-b9dd-2a4ab2286179 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:10.995575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff3ff315-7fff-41eb-b9dd-2a4ab2286179 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:10.996029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff3ff315-7fff-41eb-b9dd-2a4ab2286179 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:10.996799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff3ff315-7fff-41eb-b9dd-2a4ab2286179 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:10.997235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff3ff315-7fff-41eb-b9dd-2a4ab2286179 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:11.027564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ff3ff315-7fff-41eb-b9dd-2a4ab2286179 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044490 May 25 16:35:11.028152 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 529/1075] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:10 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:12.041708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b16d6a14-c6e9-42cc-ae3f-84aeeff0feb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:12.042849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b16d6a14-c6e9-42cc-ae3f-84aeeff0feb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:12.052370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b16d6a14-c6e9-42cc-ae3f-84aeeff0feb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:12.052818 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b16d6a14-c6e9-42cc-ae3f-84aeeff0feb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:12.053662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b16d6a14-c6e9-42cc-ae3f-84aeeff0feb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:12.054101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b16d6a14-c6e9-42cc-ae3f-84aeeff0feb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:12.083989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b16d6a14-c6e9-42cc-ae3f-84aeeff0feb2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048051 May 25 16:35:12.084851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 547/1076] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:12 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:13.095815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-061d096d-eb39-4ac4-85ed-8cf8910dd6a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:13.096782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-061d096d-eb39-4ac4-85ed-8cf8910dd6a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:13.104028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-061d096d-eb39-4ac4-85ed-8cf8910dd6a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:13.104487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-061d096d-eb39-4ac4-85ed-8cf8910dd6a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:13.105259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-061d096d-eb39-4ac4-85ed-8cf8910dd6a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:13.105697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-061d096d-eb39-4ac4-85ed-8cf8910dd6a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:13.136128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-061d096d-eb39-4ac4-85ed-8cf8910dd6a8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044019 May 25 16:35:13.136731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 530/1077] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:13 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:14.146967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-458be283-498b-4e28-b1d4-d9eaefa3a592 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:14.147957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-458be283-498b-4e28-b1d4-d9eaefa3a592 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:14.155055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-458be283-498b-4e28-b1d4-d9eaefa3a592 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:14.155528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-458be283-498b-4e28-b1d4-d9eaefa3a592 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:14.156284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-458be283-498b-4e28-b1d4-d9eaefa3a592 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:14.156714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-458be283-498b-4e28-b1d4-d9eaefa3a592 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:14.188846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-458be283-498b-4e28-b1d4-d9eaefa3a592 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045739 May 25 16:35:14.189429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 548/1078] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:14 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:15.201001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-768623a5-051f-49a0-98fb-2b7302e28d46 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:15.201933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-768623a5-051f-49a0-98fb-2b7302e28d46 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:15.208815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-768623a5-051f-49a0-98fb-2b7302e28d46 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:15.209267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-768623a5-051f-49a0-98fb-2b7302e28d46 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:15.210027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-768623a5-051f-49a0-98fb-2b7302e28d46 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:15.210455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-768623a5-051f-49a0-98fb-2b7302e28d46 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:15.240660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-768623a5-051f-49a0-98fb-2b7302e28d46 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043605 May 25 16:35:15.241226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 531/1079] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:15 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:16.251793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8bde509e-0f97-4695-8bd4-e82fd9e30eb5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:16.252742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8bde509e-0f97-4695-8bd4-e82fd9e30eb5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:16.259607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bde509e-0f97-4695-8bd4-e82fd9e30eb5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:16.260060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bde509e-0f97-4695-8bd4-e82fd9e30eb5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:16.260828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bde509e-0f97-4695-8bd4-e82fd9e30eb5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:16.261264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bde509e-0f97-4695-8bd4-e82fd9e30eb5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:16.291573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8bde509e-0f97-4695-8bd4-e82fd9e30eb5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043766 May 25 16:35:16.292162 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 549/1080] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:16 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:17.306403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-823a4a7c-a9bc-4b85-bed8-6ac1272d353a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:17.307368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-823a4a7c-a9bc-4b85-bed8-6ac1272d353a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:17.314222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-823a4a7c-a9bc-4b85-bed8-6ac1272d353a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:17.314676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-823a4a7c-a9bc-4b85-bed8-6ac1272d353a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:17.315490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-823a4a7c-a9bc-4b85-bed8-6ac1272d353a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:17.315925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-823a4a7c-a9bc-4b85-bed8-6ac1272d353a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:17.346152 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-823a4a7c-a9bc-4b85-bed8-6ac1272d353a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045612 May 25 16:35:17.346720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 532/1081] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:17 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:18.358699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6d261650-190e-415c-ad66-5e0cb5f96cd8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:18.359752 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6d261650-190e-415c-ad66-5e0cb5f96cd8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:18.366894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d261650-190e-415c-ad66-5e0cb5f96cd8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:18.367469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d261650-190e-415c-ad66-5e0cb5f96cd8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:18.368300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d261650-190e-415c-ad66-5e0cb5f96cd8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:18.368808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d261650-190e-415c-ad66-5e0cb5f96cd8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:18.413832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6d261650-190e-415c-ad66-5e0cb5f96cd8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059227 May 25 16:35:18.418505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 550/1082] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:18 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:19.428059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-16afd008-fc82-4f6e-a88d-1c4eefc41537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:19.429414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-16afd008-fc82-4f6e-a88d-1c4eefc41537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:19.436560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-16afd008-fc82-4f6e-a88d-1c4eefc41537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:19.436983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-16afd008-fc82-4f6e-a88d-1c4eefc41537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:19.437787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-16afd008-fc82-4f6e-a88d-1c4eefc41537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:19.438226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-16afd008-fc82-4f6e-a88d-1c4eefc41537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:19.468803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-16afd008-fc82-4f6e-a88d-1c4eefc41537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046195 May 25 16:35:19.469378 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 533/1083] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:19 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:20.485325 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d87c8f79-39ca-4765-87ff-922b1c2f2b65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:20.486830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d87c8f79-39ca-4765-87ff-922b1c2f2b65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:20.495788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d87c8f79-39ca-4765-87ff-922b1c2f2b65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:20.496240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d87c8f79-39ca-4765-87ff-922b1c2f2b65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:20.497034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d87c8f79-39ca-4765-87ff-922b1c2f2b65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:20.497472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d87c8f79-39ca-4765-87ff-922b1c2f2b65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:20.528140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d87c8f79-39ca-4765-87ff-922b1c2f2b65 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049530 May 25 16:35:20.529064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 551/1084] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:20 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:21.543297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8a408556-0be4-4753-8b00-b65d3289c8f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:21.544392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8a408556-0be4-4753-8b00-b65d3289c8f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:21.552536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a408556-0be4-4753-8b00-b65d3289c8f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:21.553068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a408556-0be4-4753-8b00-b65d3289c8f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:21.553974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a408556-0be4-4753-8b00-b65d3289c8f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:21.554491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a408556-0be4-4753-8b00-b65d3289c8f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:21.586982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8a408556-0be4-4753-8b00-b65d3289c8f4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049407 May 25 16:35:21.587667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 534/1085] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:21 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:22.603301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b4fa1834-376b-4112-9a70-a5cbb778a1b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:22.604843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b4fa1834-376b-4112-9a70-a5cbb778a1b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:22.615444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b4fa1834-376b-4112-9a70-a5cbb778a1b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:22.616177 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b4fa1834-376b-4112-9a70-a5cbb778a1b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:22.617480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b4fa1834-376b-4112-9a70-a5cbb778a1b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:22.617920 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b4fa1834-376b-4112-9a70-a5cbb778a1b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:22.648341 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b4fa1834-376b-4112-9a70-a5cbb778a1b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051594 May 25 16:35:22.648962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 552/1086] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:22 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:23.663136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1b4e4655-4d7f-4707-a4e2-2e38a235baa4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:23.664467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1b4e4655-4d7f-4707-a4e2-2e38a235baa4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:23.671705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1b4e4655-4d7f-4707-a4e2-2e38a235baa4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:23.672150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1b4e4655-4d7f-4707-a4e2-2e38a235baa4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:23.672927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1b4e4655-4d7f-4707-a4e2-2e38a235baa4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:23.673355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1b4e4655-4d7f-4707-a4e2-2e38a235baa4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:23.703693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1b4e4655-4d7f-4707-a4e2-2e38a235baa4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047081 May 25 16:35:23.704229 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 535/1087] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:23 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:24.719860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0c8ba601-4db0-4ee2-adec-ffbfeec907b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:24.721343 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0c8ba601-4db0-4ee2-adec-ffbfeec907b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:24.732893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c8ba601-4db0-4ee2-adec-ffbfeec907b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:24.733571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c8ba601-4db0-4ee2-adec-ffbfeec907b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:24.734751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c8ba601-4db0-4ee2-adec-ffbfeec907b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:24.735605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c8ba601-4db0-4ee2-adec-ffbfeec907b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:24.767642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0c8ba601-4db0-4ee2-adec-ffbfeec907b8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054422 May 25 16:35:24.768199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 553/1088] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:24 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:25.783827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d65a9266-d4c7-431c-b75b-cf9d6c343c55 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:25.785317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d65a9266-d4c7-431c-b75b-cf9d6c343c55 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:25.795281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d65a9266-d4c7-431c-b75b-cf9d6c343c55 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:25.795925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d65a9266-d4c7-431c-b75b-cf9d6c343c55 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:25.797024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d65a9266-d4c7-431c-b75b-cf9d6c343c55 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:25.797681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d65a9266-d4c7-431c-b75b-cf9d6c343c55 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:25.829737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d65a9266-d4c7-431c-b75b-cf9d6c343c55 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052606 May 25 16:35:25.830314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 536/1089] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:25 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:35:26.841531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-27d49d0c-23c6-4db2-9cd9-bfc153629c80 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:26.842461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-27d49d0c-23c6-4db2-9cd9-bfc153629c80 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:26.849562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27d49d0c-23c6-4db2-9cd9-bfc153629c80 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:26.850128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27d49d0c-23c6-4db2-9cd9-bfc153629c80 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:26.851089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27d49d0c-23c6-4db2-9cd9-bfc153629c80 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:26.851319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27d49d0c-23c6-4db2-9cd9-bfc153629c80 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:26.880885 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-27d49d0c-23c6-4db2-9cd9-bfc153629c80 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043417 May 25 16:35:26.881670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 554/1090] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:26 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:27.897385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-78078972-48d6-48fb-9215-f966400b79bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:27.898820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-78078972-48d6-48fb-9215-f966400b79bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:27.909630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78078972-48d6-48fb-9215-f966400b79bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:27.909841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78078972-48d6-48fb-9215-f966400b79bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:27.910851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78078972-48d6-48fb-9215-f966400b79bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:27.911337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78078972-48d6-48fb-9215-f966400b79bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:27.943657 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-78078972-48d6-48fb-9215-f966400b79bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053209 May 25 16:35:27.944542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 537/1091] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:27 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:28.960121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3ffd797c-d90a-4242-8136-73251f2ee6ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:28.961616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3ffd797c-d90a-4242-8136-73251f2ee6ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:28.973017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ffd797c-d90a-4242-8136-73251f2ee6ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:28.973934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ffd797c-d90a-4242-8136-73251f2ee6ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:28.975118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ffd797c-d90a-4242-8136-73251f2ee6ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:28.976152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ffd797c-d90a-4242-8136-73251f2ee6ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:29.006609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3ffd797c-d90a-4242-8136-73251f2ee6ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053118 May 25 16:35:29.007206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 555/1092] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:28 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:30.021009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-31a37048-f227-44b2-87b1-c4c4f2e6857e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:30.021974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-31a37048-f227-44b2-87b1-c4c4f2e6857e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:30.029520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31a37048-f227-44b2-87b1-c4c4f2e6857e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:30.029950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31a37048-f227-44b2-87b1-c4c4f2e6857e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:30.030749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31a37048-f227-44b2-87b1-c4c4f2e6857e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:30.031201 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31a37048-f227-44b2-87b1-c4c4f2e6857e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:30.072431 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-31a37048-f227-44b2-87b1-c4c4f2e6857e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057595 May 25 16:35:30.073015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 538/1093] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:30 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:31.083598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9b06a8e1-595c-4a7c-bf3e-79b11eb7570e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:31.084529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9b06a8e1-595c-4a7c-bf3e-79b11eb7570e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:31.091644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b06a8e1-595c-4a7c-bf3e-79b11eb7570e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:31.092104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b06a8e1-595c-4a7c-bf3e-79b11eb7570e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:31.092858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b06a8e1-595c-4a7c-bf3e-79b11eb7570e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:31.093296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b06a8e1-595c-4a7c-bf3e-79b11eb7570e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:31.126696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9b06a8e1-595c-4a7c-bf3e-79b11eb7570e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047053 May 25 16:35:31.127462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 556/1094] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:31 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:32.142011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-21ef12a5-bafc-48d6-a7a7-12e3aac2491c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:32.144505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-21ef12a5-bafc-48d6-a7a7-12e3aac2491c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:32.156160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21ef12a5-bafc-48d6-a7a7-12e3aac2491c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:32.157367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21ef12a5-bafc-48d6-a7a7-12e3aac2491c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:32.158633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21ef12a5-bafc-48d6-a7a7-12e3aac2491c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:32.160136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21ef12a5-bafc-48d6-a7a7-12e3aac2491c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:32.192378 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-21ef12a5-bafc-48d6-a7a7-12e3aac2491c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056917 May 25 16:35:32.192941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 539/1095] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:32 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:33.208718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c4f816c2-dd07-40fd-92a4-28dd77557194 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:33.210224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c4f816c2-dd07-40fd-92a4-28dd77557194 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:33.217581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4f816c2-dd07-40fd-92a4-28dd77557194 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:33.218032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4f816c2-dd07-40fd-92a4-28dd77557194 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:33.218792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4f816c2-dd07-40fd-92a4-28dd77557194 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:33.219252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4f816c2-dd07-40fd-92a4-28dd77557194 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:33.249301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c4f816c2-dd07-40fd-92a4-28dd77557194 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047504 May 25 16:35:33.249981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 557/1096] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:33 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:33.622003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:35:33.622191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:35:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:35:34.264855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a8014d88-933f-47a9-bedc-bf5bc6415858 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:34.266341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a8014d88-933f-47a9-bedc-bf5bc6415858 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:34.277447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8014d88-933f-47a9-bedc-bf5bc6415858 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:34.278203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8014d88-933f-47a9-bedc-bf5bc6415858 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:34.279538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8014d88-933f-47a9-bedc-bf5bc6415858 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:34.280252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8014d88-933f-47a9-bedc-bf5bc6415858 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:34.314613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a8014d88-933f-47a9-bedc-bf5bc6415858 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056652 May 25 16:35:34.315182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 540/1097] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:34 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:35.329760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ee538249-0501-403f-8066-e682a6ad514f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:35.331469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ee538249-0501-403f-8066-e682a6ad514f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:35.340181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee538249-0501-403f-8066-e682a6ad514f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:35.340631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee538249-0501-403f-8066-e682a6ad514f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:35.341399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee538249-0501-403f-8066-e682a6ad514f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:35.341834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee538249-0501-403f-8066-e682a6ad514f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:35.372593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ee538249-0501-403f-8066-e682a6ad514f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049632 May 25 16:35:35.373160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 558/1098] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:35 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:36.386689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-72a74f50-3e6b-4f7f-9799-eda360c2ce21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:36.387909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-72a74f50-3e6b-4f7f-9799-eda360c2ce21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:36.396324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72a74f50-3e6b-4f7f-9799-eda360c2ce21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:36.396892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72a74f50-3e6b-4f7f-9799-eda360c2ce21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:36.397834 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72a74f50-3e6b-4f7f-9799-eda360c2ce21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:36.398380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-72a74f50-3e6b-4f7f-9799-eda360c2ce21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:36.433694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-72a74f50-3e6b-4f7f-9799-eda360c2ce21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052556 May 25 16:35:36.434226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 541/1099] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:36 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:37.448999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d92a847e-ca01-4074-8f5b-127ca8796544 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:37.450266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d92a847e-ca01-4074-8f5b-127ca8796544 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:37.459643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d92a847e-ca01-4074-8f5b-127ca8796544 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:37.460103 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d92a847e-ca01-4074-8f5b-127ca8796544 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:37.461067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d92a847e-ca01-4074-8f5b-127ca8796544 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:37.461504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d92a847e-ca01-4074-8f5b-127ca8796544 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:37.491545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d92a847e-ca01-4074-8f5b-127ca8796544 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048850 May 25 16:35:37.492233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 559/1100] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:37 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:38.505549 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0fece863-e3d2-44e3-bba7-ba1528480bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:38.506610 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0fece863-e3d2-44e3-bba7-ba1528480bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:38.513945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0fece863-e3d2-44e3-bba7-ba1528480bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:38.514392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0fece863-e3d2-44e3-bba7-ba1528480bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:38.515181 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0fece863-e3d2-44e3-bba7-ba1528480bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:38.515628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0fece863-e3d2-44e3-bba7-ba1528480bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:38.546333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0fece863-e3d2-44e3-bba7-ba1528480bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046456 May 25 16:35:38.546912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 542/1101] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:38 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:39.560085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6b418c32-6d9a-472a-b11e-7196e11c8842 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:39.561269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6b418c32-6d9a-472a-b11e-7196e11c8842 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:39.568842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b418c32-6d9a-472a-b11e-7196e11c8842 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:39.569489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b418c32-6d9a-472a-b11e-7196e11c8842 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:39.570957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b418c32-6d9a-472a-b11e-7196e11c8842 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:39.571424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b418c32-6d9a-472a-b11e-7196e11c8842 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:39.600836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6b418c32-6d9a-472a-b11e-7196e11c8842 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045504 May 25 16:35:39.604667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 560/1102] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:39 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:40.616847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-542e7427-3aa5-4a55-bae6-0246a3acac9a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:40.617816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-542e7427-3aa5-4a55-bae6-0246a3acac9a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:40.620338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:35:40.626509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-542e7427-3aa5-4a55-bae6-0246a3acac9a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:40.626961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-542e7427-3aa5-4a55-bae6-0246a3acac9a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:40.627790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-542e7427-3aa5-4a55-bae6-0246a3acac9a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:40.628174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-542e7427-3aa5-4a55-bae6-0246a3acac9a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:40.630833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=90 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:35:40.659113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-542e7427-3aa5-4a55-bae6-0246a3acac9a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049072 May 25 16:35:40.659906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 543/1103] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:40 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:41.675510 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5946910a-be06-4887-bb36-5f89a4363181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:41.677002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5946910a-be06-4887-bb36-5f89a4363181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:41.680957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=30 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:35:41.690611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5946910a-be06-4887-bb36-5f89a4363181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:41.691407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5946910a-be06-4887-bb36-5f89a4363181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:41.692663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5946910a-be06-4887-bb36-5f89a4363181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:41.693389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5946910a-be06-4887-bb36-5f89a4363181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:41.696770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=90 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:35:41.725734 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5946910a-be06-4887-bb36-5f89a4363181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056992 May 25 16:35:41.726355 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 561/1104] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:41 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:42.741956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a66496cc-bfa3-4f43-afb6-a205d273fa78 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:42.743022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a66496cc-bfa3-4f43-afb6-a205d273fa78 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:42.750062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a66496cc-bfa3-4f43-afb6-a205d273fa78 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:42.750515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a66496cc-bfa3-4f43-afb6-a205d273fa78 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:42.751292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a66496cc-bfa3-4f43-afb6-a205d273fa78 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:42.751719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a66496cc-bfa3-4f43-afb6-a205d273fa78 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:42.781105 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a66496cc-bfa3-4f43-afb6-a205d273fa78 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045897 May 25 16:35:42.781685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 544/1105] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:42 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:43.797333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fd6f5045-8787-44e3-8bf8-e2197d1fcce6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:43.799672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fd6f5045-8787-44e3-8bf8-e2197d1fcce6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:43.808016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd6f5045-8787-44e3-8bf8-e2197d1fcce6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:43.808506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd6f5045-8787-44e3-8bf8-e2197d1fcce6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:43.809287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd6f5045-8787-44e3-8bf8-e2197d1fcce6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:43.809709 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd6f5045-8787-44e3-8bf8-e2197d1fcce6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:43.843634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fd6f5045-8787-44e3-8bf8-e2197d1fcce6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053238 May 25 16:35:43.844202 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 562/1106] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:43 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:44.050486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:35:44.050983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:35:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:35:44.859357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9e69596f-a950-47e3-ad96-853c6869d741 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:44.860854 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9e69596f-a950-47e3-ad96-853c6869d741 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:44.872208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e69596f-a950-47e3-ad96-853c6869d741 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:44.872924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e69596f-a950-47e3-ad96-853c6869d741 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:44.874835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e69596f-a950-47e3-ad96-853c6869d741 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:44.874835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e69596f-a950-47e3-ad96-853c6869d741 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:44.907665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9e69596f-a950-47e3-ad96-853c6869d741 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054847 May 25 16:35:44.908225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 545/1107] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:44 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:45.923983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-437567f0-21c3-4359-ae80-6049d09f07c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:45.925472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-437567f0-21c3-4359-ae80-6049d09f07c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:45.934008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-437567f0-21c3-4359-ae80-6049d09f07c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:45.934469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-437567f0-21c3-4359-ae80-6049d09f07c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:45.935272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-437567f0-21c3-4359-ae80-6049d09f07c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:45.935809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-437567f0-21c3-4359-ae80-6049d09f07c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:45.965984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-437567f0-21c3-4359-ae80-6049d09f07c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048650 May 25 16:35:45.966745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 563/1108] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:45 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:46.981206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b359f4a0-71a7-494a-91b5-15471628b93a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:46.982705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b359f4a0-71a7-494a-91b5-15471628b93a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:46.993647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b359f4a0-71a7-494a-91b5-15471628b93a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:46.994559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b359f4a0-71a7-494a-91b5-15471628b93a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:46.995663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b359f4a0-71a7-494a-91b5-15471628b93a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:46.996384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b359f4a0-71a7-494a-91b5-15471628b93a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:47.031372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b359f4a0-71a7-494a-91b5-15471628b93a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056975 May 25 16:35:47.035291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 546/1109] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:46 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:48.048090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-216c83eb-804e-4987-8ce6-b77fe71a1cc4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:48.049654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-216c83eb-804e-4987-8ce6-b77fe71a1cc4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:48.057798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-216c83eb-804e-4987-8ce6-b77fe71a1cc4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:48.058249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-216c83eb-804e-4987-8ce6-b77fe71a1cc4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:48.059011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-216c83eb-804e-4987-8ce6-b77fe71a1cc4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:48.059471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-216c83eb-804e-4987-8ce6-b77fe71a1cc4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:48.089971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-216c83eb-804e-4987-8ce6-b77fe71a1cc4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048811 May 25 16:35:48.090564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 564/1110] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:48 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:49.102534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-120f8e1c-f66e-4ed9-8441-1b86a8416dd2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:49.103480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-120f8e1c-f66e-4ed9-8441-1b86a8416dd2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:49.110412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120f8e1c-f66e-4ed9-8441-1b86a8416dd2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:49.110851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120f8e1c-f66e-4ed9-8441-1b86a8416dd2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:49.111637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120f8e1c-f66e-4ed9-8441-1b86a8416dd2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:49.112068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120f8e1c-f66e-4ed9-8441-1b86a8416dd2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:49.142236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-120f8e1c-f66e-4ed9-8441-1b86a8416dd2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043923 May 25 16:35:49.142801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 547/1111] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:49 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:50.154044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ed4900fa-c821-4c53-92f3-3c4058e7b8ac tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:50.154967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ed4900fa-c821-4c53-92f3-3c4058e7b8ac tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:50.162007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed4900fa-c821-4c53-92f3-3c4058e7b8ac tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:50.162452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed4900fa-c821-4c53-92f3-3c4058e7b8ac tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:50.163238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed4900fa-c821-4c53-92f3-3c4058e7b8ac tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:50.163772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed4900fa-c821-4c53-92f3-3c4058e7b8ac tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:50.195345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ed4900fa-c821-4c53-92f3-3c4058e7b8ac tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045310 May 25 16:35:50.195909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 565/1112] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:50 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:51.052615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:35:51.053489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:35:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:35:51.211135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fa3555e4-a32d-4d4e-81ae-a6adc7ff3857 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:51.212193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fa3555e4-a32d-4d4e-81ae-a6adc7ff3857 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:51.219901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa3555e4-a32d-4d4e-81ae-a6adc7ff3857 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:51.220403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa3555e4-a32d-4d4e-81ae-a6adc7ff3857 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:51.221256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa3555e4-a32d-4d4e-81ae-a6adc7ff3857 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:51.221743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa3555e4-a32d-4d4e-81ae-a6adc7ff3857 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:51.254767 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fa3555e4-a32d-4d4e-81ae-a6adc7ff3857 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050184 May 25 16:35:51.255340 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 548/1113] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:51 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:52.269940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4c1aa5e6-62c1-44d8-b5a6-0596d695b428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:52.271495 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4c1aa5e6-62c1-44d8-b5a6-0596d695b428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:52.282751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c1aa5e6-62c1-44d8-b5a6-0596d695b428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:52.283561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c1aa5e6-62c1-44d8-b5a6-0596d695b428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:52.284827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c1aa5e6-62c1-44d8-b5a6-0596d695b428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:52.285555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c1aa5e6-62c1-44d8-b5a6-0596d695b428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:52.318265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4c1aa5e6-62c1-44d8-b5a6-0596d695b428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055034 May 25 16:35:52.318830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 566/1114] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:52 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:53.339406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-675a5e69-7ca8-4c1e-8649-a8b0c6cde09e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:53.340496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-675a5e69-7ca8-4c1e-8649-a8b0c6cde09e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:53.349846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-675a5e69-7ca8-4c1e-8649-a8b0c6cde09e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:53.350414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-675a5e69-7ca8-4c1e-8649-a8b0c6cde09e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:53.351625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-675a5e69-7ca8-4c1e-8649-a8b0c6cde09e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:53.352139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-675a5e69-7ca8-4c1e-8649-a8b0c6cde09e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:53.384919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-675a5e69-7ca8-4c1e-8649-a8b0c6cde09e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055281 May 25 16:35:53.385490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 549/1115] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:53 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:54.399924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a43b8de7-f384-4108-9a8d-bae9dad23561 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:54.401418 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a43b8de7-f384-4108-9a8d-bae9dad23561 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:54.412405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a43b8de7-f384-4108-9a8d-bae9dad23561 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:54.413164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a43b8de7-f384-4108-9a8d-bae9dad23561 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:54.414139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a43b8de7-f384-4108-9a8d-bae9dad23561 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:54.414770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a43b8de7-f384-4108-9a8d-bae9dad23561 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:54.444595 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a43b8de7-f384-4108-9a8d-bae9dad23561 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051332 May 25 16:35:54.445161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 567/1116] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:54 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:55.458645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4fe1fcd5-a735-4780-ac43-e7700dd37c41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:55.459930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4fe1fcd5-a735-4780-ac43-e7700dd37c41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:55.467912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fe1fcd5-a735-4780-ac43-e7700dd37c41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:55.468444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fe1fcd5-a735-4780-ac43-e7700dd37c41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:55.469399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fe1fcd5-a735-4780-ac43-e7700dd37c41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:55.469801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fe1fcd5-a735-4780-ac43-e7700dd37c41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:55.502582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4fe1fcd5-a735-4780-ac43-e7700dd37c41 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049592 May 25 16:35:55.503180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 550/1117] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:55 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:56.518183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b8809b51-c760-4dcf-ae84-dd34580bd2dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:56.519545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b8809b51-c760-4dcf-ae84-dd34580bd2dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:56.527739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b8809b51-c760-4dcf-ae84-dd34580bd2dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:56.528189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b8809b51-c760-4dcf-ae84-dd34580bd2dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:56.529014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b8809b51-c760-4dcf-ae84-dd34580bd2dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:56.529478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b8809b51-c760-4dcf-ae84-dd34580bd2dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:56.559647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b8809b51-c760-4dcf-ae84-dd34580bd2dc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048249 May 25 16:35:56.560217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 568/1118] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:56 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:57.573269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cf826779-f742-4dc5-aea1-85522050279b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:57.574702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cf826779-f742-4dc5-aea1-85522050279b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:57.582071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cf826779-f742-4dc5-aea1-85522050279b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:57.583222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cf826779-f742-4dc5-aea1-85522050279b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:57.584232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cf826779-f742-4dc5-aea1-85522050279b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:57.584526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cf826779-f742-4dc5-aea1-85522050279b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:57.616073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cf826779-f742-4dc5-aea1-85522050279b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047533 May 25 16:35:57.616617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 551/1119] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:57 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:58.632019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d417562e-65b3-4ec7-839d-214a76146f33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:58.633643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d417562e-65b3-4ec7-839d-214a76146f33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:58.644564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d417562e-65b3-4ec7-839d-214a76146f33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:58.645316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d417562e-65b3-4ec7-839d-214a76146f33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:58.646581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d417562e-65b3-4ec7-839d-214a76146f33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:58.647346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d417562e-65b3-4ec7-839d-214a76146f33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:58.677960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d417562e-65b3-4ec7-839d-214a76146f33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052646 May 25 16:35:58.678536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 569/1120] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:58 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:35:59.694270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-63a73a35-a9e0-4cdc-9e4b-25bc796e9304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:35:59.695453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-63a73a35-a9e0-4cdc-9e4b-25bc796e9304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:35:59.702892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-63a73a35-a9e0-4cdc-9e4b-25bc796e9304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:59.703730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-63a73a35-a9e0-4cdc-9e4b-25bc796e9304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:59.704430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-63a73a35-a9e0-4cdc-9e4b-25bc796e9304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:35:59.704946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-63a73a35-a9e0-4cdc-9e4b-25bc796e9304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:35:59.738988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-63a73a35-a9e0-4cdc-9e4b-25bc796e9304 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051601 May 25 16:35:59.739583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 552/1121] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:35:59 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:00.757047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f3fd91bd-5726-4062-aee2-3e4aa9df6ee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:00.757047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f3fd91bd-5726-4062-aee2-3e4aa9df6ee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:00.768926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3fd91bd-5726-4062-aee2-3e4aa9df6ee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:00.770013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3fd91bd-5726-4062-aee2-3e4aa9df6ee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:00.771317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3fd91bd-5726-4062-aee2-3e4aa9df6ee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:00.772037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3fd91bd-5726-4062-aee2-3e4aa9df6ee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:00.810301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f3fd91bd-5726-4062-aee2-3e4aa9df6ee6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061822 May 25 16:36:00.810917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 570/1122] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:00 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:01.824780 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3a5bc6a9-8014-41df-a2f7-189d3778b40a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:01.825829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3a5bc6a9-8014-41df-a2f7-189d3778b40a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:01.834153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a5bc6a9-8014-41df-a2f7-189d3778b40a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:01.834593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a5bc6a9-8014-41df-a2f7-189d3778b40a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:01.835480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a5bc6a9-8014-41df-a2f7-189d3778b40a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:01.835910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a5bc6a9-8014-41df-a2f7-189d3778b40a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:01.866355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3a5bc6a9-8014-41df-a2f7-189d3778b40a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047554 May 25 16:36:01.866915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 553/1123] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:01 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:02.882313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-eec7b5d7-caa1-4d61-a32b-a4002f7ad331 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:02.883379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-eec7b5d7-caa1-4d61-a32b-a4002f7ad331 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:02.891903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eec7b5d7-caa1-4d61-a32b-a4002f7ad331 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:02.892582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eec7b5d7-caa1-4d61-a32b-a4002f7ad331 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:02.893497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eec7b5d7-caa1-4d61-a32b-a4002f7ad331 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:02.893987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eec7b5d7-caa1-4d61-a32b-a4002f7ad331 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:02.927816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-eec7b5d7-caa1-4d61-a32b-a4002f7ad331 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052209 May 25 16:36:02.928366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 571/1124] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:02 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:03.939849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-47c46d9e-554d-4601-94d6-12a0f3308bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:03.940741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-47c46d9e-554d-4601-94d6-12a0f3308bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:03.948231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47c46d9e-554d-4601-94d6-12a0f3308bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:03.948446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47c46d9e-554d-4601-94d6-12a0f3308bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:03.949208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47c46d9e-554d-4601-94d6-12a0f3308bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:03.949647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47c46d9e-554d-4601-94d6-12a0f3308bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:03.979895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-47c46d9e-554d-4601-94d6-12a0f3308bc0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044121 May 25 16:36:03.980469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 554/1125] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:03 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:04.996249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bd48fd3b-4f35-49e4-b56a-423c95644d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:04.997632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bd48fd3b-4f35-49e4-b56a-423c95644d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:05.008054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd48fd3b-4f35-49e4-b56a-423c95644d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:05.008602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd48fd3b-4f35-49e4-b56a-423c95644d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:05.009247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd48fd3b-4f35-49e4-b56a-423c95644d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:05.009678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd48fd3b-4f35-49e4-b56a-423c95644d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:05.039461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bd48fd3b-4f35-49e4-b56a-423c95644d2f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049992 May 25 16:36:05.040345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 572/1126] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:04 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:06.055240 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5663c1c8-f870-451a-9299-5a25b061401f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:06.056395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5663c1c8-f870-451a-9299-5a25b061401f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:06.064142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5663c1c8-f870-451a-9299-5a25b061401f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:06.064614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5663c1c8-f870-451a-9299-5a25b061401f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:06.065413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5663c1c8-f870-451a-9299-5a25b061401f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:06.065860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5663c1c8-f870-451a-9299-5a25b061401f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:06.096459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5663c1c8-f870-451a-9299-5a25b061401f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047465 May 25 16:36:06.097022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 555/1127] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:06 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:07.108518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ca654847-2254-4ad3-b41b-1722dff860f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:07.109453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ca654847-2254-4ad3-b41b-1722dff860f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:07.116573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ca654847-2254-4ad3-b41b-1722dff860f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:07.117029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ca654847-2254-4ad3-b41b-1722dff860f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:07.117784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ca654847-2254-4ad3-b41b-1722dff860f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:07.118476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ca654847-2254-4ad3-b41b-1722dff860f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:07.149437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ca654847-2254-4ad3-b41b-1722dff860f2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044903 May 25 16:36:07.150030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 573/1128] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:07 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:08.161269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-30288e58-15da-40a9-9a96-ae5a5180abfe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:08.162204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-30288e58-15da-40a9-9a96-ae5a5180abfe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:08.169823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30288e58-15da-40a9-9a96-ae5a5180abfe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:08.170277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30288e58-15da-40a9-9a96-ae5a5180abfe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:08.171044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30288e58-15da-40a9-9a96-ae5a5180abfe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:08.171750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30288e58-15da-40a9-9a96-ae5a5180abfe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:08.201656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-30288e58-15da-40a9-9a96-ae5a5180abfe tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044338 May 25 16:36:08.202558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 556/1129] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:08 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:09.217953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f1bfaa9e-0010-4169-b57f-84bef85dc278 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:09.219357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f1bfaa9e-0010-4169-b57f-84bef85dc278 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:09.230459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f1bfaa9e-0010-4169-b57f-84bef85dc278 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:09.231613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f1bfaa9e-0010-4169-b57f-84bef85dc278 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:09.232529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f1bfaa9e-0010-4169-b57f-84bef85dc278 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:09.233257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f1bfaa9e-0010-4169-b57f-84bef85dc278 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:09.266901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f1bfaa9e-0010-4169-b57f-84bef85dc278 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055752 May 25 16:36:09.267571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 574/1130] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:09 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:10.282464 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ea5181f7-1a32-4dd5-a231-6c43b9c9734c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:10.283981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ea5181f7-1a32-4dd5-a231-6c43b9c9734c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:10.292590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea5181f7-1a32-4dd5-a231-6c43b9c9734c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:10.293049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea5181f7-1a32-4dd5-a231-6c43b9c9734c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:10.293808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea5181f7-1a32-4dd5-a231-6c43b9c9734c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:10.294201 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea5181f7-1a32-4dd5-a231-6c43b9c9734c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:10.324295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ea5181f7-1a32-4dd5-a231-6c43b9c9734c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048542 May 25 16:36:10.324867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 557/1131] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:10 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:11.341048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-21f5e8f1-e12b-47f4-8005-aaa67ccbc68d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:11.342573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-21f5e8f1-e12b-47f4-8005-aaa67ccbc68d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:11.353973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21f5e8f1-e12b-47f4-8005-aaa67ccbc68d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:11.354875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21f5e8f1-e12b-47f4-8005-aaa67ccbc68d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:11.356201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21f5e8f1-e12b-47f4-8005-aaa67ccbc68d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:11.356935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21f5e8f1-e12b-47f4-8005-aaa67ccbc68d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:11.390509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-21f5e8f1-e12b-47f4-8005-aaa67ccbc68d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056948 May 25 16:36:11.391098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 575/1132] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:11 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:12.406611 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b5b90278-269e-4026-b470-ff4fc1ecbeb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:12.408027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b5b90278-269e-4026-b470-ff4fc1ecbeb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:12.411669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:36:12.418249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5b90278-269e-4026-b470-ff4fc1ecbeb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:12.418707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5b90278-269e-4026-b470-ff4fc1ecbeb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:12.419500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5b90278-269e-4026-b470-ff4fc1ecbeb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:12.419944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5b90278-269e-4026-b470-ff4fc1ecbeb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:12.422664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=90 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:36:12.451637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b5b90278-269e-4026-b470-ff4fc1ecbeb8 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051896 May 25 16:36:12.452427 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 558/1133] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:12 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:13.468272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8da313b2-38cb-42e2-a667-975c66253b34 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:13.469868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8da313b2-38cb-42e2-a667-975c66253b34 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:13.473541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=30 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:36:13.483376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8da313b2-38cb-42e2-a667-975c66253b34 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:13.484171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8da313b2-38cb-42e2-a667-975c66253b34 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:13.485437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8da313b2-38cb-42e2-a667-975c66253b34 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:13.486157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8da313b2-38cb-42e2-a667-975c66253b34 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:13.490831 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=90 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:36:13.518439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8da313b2-38cb-42e2-a667-975c66253b34 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057142 May 25 16:36:13.519307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 576/1134] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:13 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:14.534337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4e7b8996-23e3-45ac-a386-66b2f1f6d422 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:14.535428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4e7b8996-23e3-45ac-a386-66b2f1f6d422 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:14.545751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4e7b8996-23e3-45ac-a386-66b2f1f6d422 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:14.546231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4e7b8996-23e3-45ac-a386-66b2f1f6d422 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:14.547591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4e7b8996-23e3-45ac-a386-66b2f1f6d422 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:14.548308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4e7b8996-23e3-45ac-a386-66b2f1f6d422 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:14.579909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4e7b8996-23e3-45ac-a386-66b2f1f6d422 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052249 May 25 16:36:14.580468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 559/1135] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:14 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:15.595984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2121f1bc-22bd-413d-8b02-5a9286ccf3c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:15.597693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2121f1bc-22bd-413d-8b02-5a9286ccf3c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:15.605465 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2121f1bc-22bd-413d-8b02-5a9286ccf3c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:15.605920 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2121f1bc-22bd-413d-8b02-5a9286ccf3c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:15.606686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2121f1bc-22bd-413d-8b02-5a9286ccf3c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:15.607124 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2121f1bc-22bd-413d-8b02-5a9286ccf3c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:15.637334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2121f1bc-22bd-413d-8b02-5a9286ccf3c7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047951 May 25 16:36:15.637890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 577/1136] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:15 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:16.653847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1802baaa-0205-455e-af26-a2dab728e0f7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:16.656127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1802baaa-0205-455e-af26-a2dab728e0f7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:16.666358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1802baaa-0205-455e-af26-a2dab728e0f7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:16.667147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1802baaa-0205-455e-af26-a2dab728e0f7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:16.668444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1802baaa-0205-455e-af26-a2dab728e0f7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:16.669300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1802baaa-0205-455e-af26-a2dab728e0f7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:16.702823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1802baaa-0205-455e-af26-a2dab728e0f7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055861 May 25 16:36:16.703550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 560/1137] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:16 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:17.718847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-68659e83-64a2-4f34-b3c7-2fb364190b06 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:17.720267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-68659e83-64a2-4f34-b3c7-2fb364190b06 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:17.728789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68659e83-64a2-4f34-b3c7-2fb364190b06 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:17.729245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68659e83-64a2-4f34-b3c7-2fb364190b06 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:17.730018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68659e83-64a2-4f34-b3c7-2fb364190b06 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:17.730455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68659e83-64a2-4f34-b3c7-2fb364190b06 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:17.759660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-68659e83-64a2-4f34-b3c7-2fb364190b06 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047568 May 25 16:36:17.760225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 578/1138] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:17 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:18.775872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7f31e24c-94da-4898-8454-2331cb9a1cd1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:18.777427 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7f31e24c-94da-4898-8454-2331cb9a1cd1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:18.789630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7f31e24c-94da-4898-8454-2331cb9a1cd1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:18.790721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7f31e24c-94da-4898-8454-2331cb9a1cd1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:18.792092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7f31e24c-94da-4898-8454-2331cb9a1cd1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:18.792778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7f31e24c-94da-4898-8454-2331cb9a1cd1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:18.827529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7f31e24c-94da-4898-8454-2331cb9a1cd1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058285 May 25 16:36:18.828531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 561/1139] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:18 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:19.842944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0e6fefc5-f372-45c7-9f5e-905220433440 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:19.844491 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0e6fefc5-f372-45c7-9f5e-905220433440 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:19.853445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e6fefc5-f372-45c7-9f5e-905220433440 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:19.853902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e6fefc5-f372-45c7-9f5e-905220433440 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:19.854658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e6fefc5-f372-45c7-9f5e-905220433440 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:19.855093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e6fefc5-f372-45c7-9f5e-905220433440 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:19.885471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0e6fefc5-f372-45c7-9f5e-905220433440 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049466 May 25 16:36:19.886060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 579/1140] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:19 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:20.902077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9dc934a6-4477-4f6d-a700-eaf3259dbacf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:20.903668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9dc934a6-4477-4f6d-a700-eaf3259dbacf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:20.913950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dc934a6-4477-4f6d-a700-eaf3259dbacf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:20.914592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dc934a6-4477-4f6d-a700-eaf3259dbacf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:20.918551 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dc934a6-4477-4f6d-a700-eaf3259dbacf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:20.919185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dc934a6-4477-4f6d-a700-eaf3259dbacf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:20.950040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9dc934a6-4477-4f6d-a700-eaf3259dbacf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054914 May 25 16:36:20.950754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 562/1141] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:20 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:21.965881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6ff29e77-1fdc-4b81-9776-616bfd337ae3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:21.967482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6ff29e77-1fdc-4b81-9776-616bfd337ae3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:21.978479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ff29e77-1fdc-4b81-9776-616bfd337ae3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:21.979252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ff29e77-1fdc-4b81-9776-616bfd337ae3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:21.980512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ff29e77-1fdc-4b81-9776-616bfd337ae3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:21.981379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ff29e77-1fdc-4b81-9776-616bfd337ae3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:22.014769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6ff29e77-1fdc-4b81-9776-616bfd337ae3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055655 May 25 16:36:22.015359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 580/1142] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:21 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:23.030193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3018a9ee-0fd5-4d46-a6fc-b202761be639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:23.031736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3018a9ee-0fd5-4d46-a6fc-b202761be639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:23.041158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3018a9ee-0fd5-4d46-a6fc-b202761be639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:23.041616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3018a9ee-0fd5-4d46-a6fc-b202761be639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:23.042351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3018a9ee-0fd5-4d46-a6fc-b202761be639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:23.042804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3018a9ee-0fd5-4d46-a6fc-b202761be639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:23.073539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3018a9ee-0fd5-4d46-a6fc-b202761be639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050065 May 25 16:36:23.074470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 563/1143] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:23 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:24.087203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-471d7f2b-8a28-4cd4-8042-6a95d0adc9ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:24.088120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-471d7f2b-8a28-4cd4-8042-6a95d0adc9ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:24.095089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-471d7f2b-8a28-4cd4-8042-6a95d0adc9ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:24.096071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-471d7f2b-8a28-4cd4-8042-6a95d0adc9ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:24.096349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-471d7f2b-8a28-4cd4-8042-6a95d0adc9ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:24.096781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-471d7f2b-8a28-4cd4-8042-6a95d0adc9ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:24.131983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-471d7f2b-8a28-4cd4-8042-6a95d0adc9ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049652 May 25 16:36:24.132526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 581/1144] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:24 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:25.143305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-30d48bab-9000-409b-aa75-52a0fa324bdd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:25.144338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-30d48bab-9000-409b-aa75-52a0fa324bdd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:25.151317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30d48bab-9000-409b-aa75-52a0fa324bdd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:25.151766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30d48bab-9000-409b-aa75-52a0fa324bdd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:25.152494 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30d48bab-9000-409b-aa75-52a0fa324bdd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:25.152946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30d48bab-9000-409b-aa75-52a0fa324bdd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:25.184993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-30d48bab-9000-409b-aa75-52a0fa324bdd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045702 May 25 16:36:25.185745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 564/1145] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:25 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:26.201225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f00fa46e-e3ea-493e-956b-6e48d993a9a3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:26.202659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f00fa46e-e3ea-493e-956b-6e48d993a9a3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:26.212087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f00fa46e-e3ea-493e-956b-6e48d993a9a3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:26.212535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f00fa46e-e3ea-493e-956b-6e48d993a9a3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:26.213768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f00fa46e-e3ea-493e-956b-6e48d993a9a3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:26.213768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f00fa46e-e3ea-493e-956b-6e48d993a9a3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:26.243437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f00fa46e-e3ea-493e-956b-6e48d993a9a3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048930 May 25 16:36:26.244001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 582/1146] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:26 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:27.259645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-607a3ba0-e21a-4dad-be2b-1f96f3af2649 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:27.261015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-607a3ba0-e21a-4dad-be2b-1f96f3af2649 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:27.272297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-607a3ba0-e21a-4dad-be2b-1f96f3af2649 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:27.272635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-607a3ba0-e21a-4dad-be2b-1f96f3af2649 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:27.273397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-607a3ba0-e21a-4dad-be2b-1f96f3af2649 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:27.273836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-607a3ba0-e21a-4dad-be2b-1f96f3af2649 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:27.306138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-607a3ba0-e21a-4dad-be2b-1f96f3af2649 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053297 May 25 16:36:27.306713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 565/1147] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:27 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:28.322768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c7c187f4-80de-4634-a517-cadd97cd7b10 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:28.324312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c7c187f4-80de-4634-a517-cadd97cd7b10 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:28.336435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7c187f4-80de-4634-a517-cadd97cd7b10 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:28.337739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7c187f4-80de-4634-a517-cadd97cd7b10 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:28.340038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7c187f4-80de-4634-a517-cadd97cd7b10 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:28.341437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7c187f4-80de-4634-a517-cadd97cd7b10 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:28.378464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c7c187f4-80de-4634-a517-cadd97cd7b10 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062527 May 25 16:36:28.379024 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 583/1148] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:28 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:29.395254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1d30f7c7-df61-47ef-bf17-c51a0aaff92b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:29.396616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1d30f7c7-df61-47ef-bf17-c51a0aaff92b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:29.404665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1d30f7c7-df61-47ef-bf17-c51a0aaff92b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:29.405106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1d30f7c7-df61-47ef-bf17-c51a0aaff92b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:29.405901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1d30f7c7-df61-47ef-bf17-c51a0aaff92b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:29.406353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1d30f7c7-df61-47ef-bf17-c51a0aaff92b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:29.435738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1d30f7c7-df61-47ef-bf17-c51a0aaff92b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048106 May 25 16:36:29.436268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 566/1149] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:29 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:30.448578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b799e362-d8e9-494e-89b0-26eb3c765f08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:30.449483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b799e362-d8e9-494e-89b0-26eb3c765f08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:30.457522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b799e362-d8e9-494e-89b0-26eb3c765f08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:30.458296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b799e362-d8e9-494e-89b0-26eb3c765f08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:30.459606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b799e362-d8e9-494e-89b0-26eb3c765f08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:30.460215 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b799e362-d8e9-494e-89b0-26eb3c765f08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:30.493137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b799e362-d8e9-494e-89b0-26eb3c765f08 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048831 May 25 16:36:30.493693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 584/1150] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:30 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:31.509442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-00de734c-e86e-4ae8-9e59-3e587769bc79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:31.510815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-00de734c-e86e-4ae8-9e59-3e587769bc79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:31.521926 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00de734c-e86e-4ae8-9e59-3e587769bc79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:31.522691 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00de734c-e86e-4ae8-9e59-3e587769bc79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:31.524006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00de734c-e86e-4ae8-9e59-3e587769bc79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:31.524741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00de734c-e86e-4ae8-9e59-3e587769bc79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:31.554973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-00de734c-e86e-4ae8-9e59-3e587769bc79 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052412 May 25 16:36:31.555772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 567/1151] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:31 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:32.570332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-02c4df55-5a5b-4380-972d-846a5f809897 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:32.571397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-02c4df55-5a5b-4380-972d-846a5f809897 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:32.578420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02c4df55-5a5b-4380-972d-846a5f809897 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:32.578869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02c4df55-5a5b-4380-972d-846a5f809897 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:32.579675 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02c4df55-5a5b-4380-972d-846a5f809897 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:32.580122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02c4df55-5a5b-4380-972d-846a5f809897 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:32.611044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-02c4df55-5a5b-4380-972d-846a5f809897 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046948 May 25 16:36:32.611628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 585/1152] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:32 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:33.626861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-86b46f40-5602-46f5-8024-3ac85a87b795 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:33.628354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-86b46f40-5602-46f5-8024-3ac85a87b795 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:33.636112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86b46f40-5602-46f5-8024-3ac85a87b795 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:33.636568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86b46f40-5602-46f5-8024-3ac85a87b795 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:33.637312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86b46f40-5602-46f5-8024-3ac85a87b795 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:33.637771 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86b46f40-5602-46f5-8024-3ac85a87b795 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:33.668225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-86b46f40-5602-46f5-8024-3ac85a87b795 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048111 May 25 16:36:33.669012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 568/1153] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:33 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:34.682870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dec191d1-04fe-4a83-9456-eb70b2e7f396 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:34.684048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dec191d1-04fe-4a83-9456-eb70b2e7f396 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:34.692984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dec191d1-04fe-4a83-9456-eb70b2e7f396 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:34.692984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dec191d1-04fe-4a83-9456-eb70b2e7f396 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:34.694085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dec191d1-04fe-4a83-9456-eb70b2e7f396 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:34.694609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dec191d1-04fe-4a83-9456-eb70b2e7f396 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:34.735972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dec191d1-04fe-4a83-9456-eb70b2e7f396 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058364 May 25 16:36:34.736529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 586/1154] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:34 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:35.754966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b45abda9-3d5e-40d7-9742-f94ad8f4072f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:35.756455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b45abda9-3d5e-40d7-9742-f94ad8f4072f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:35.763664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b45abda9-3d5e-40d7-9742-f94ad8f4072f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:35.764112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b45abda9-3d5e-40d7-9742-f94ad8f4072f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:35.766279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b45abda9-3d5e-40d7-9742-f94ad8f4072f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:35.766718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b45abda9-3d5e-40d7-9742-f94ad8f4072f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:35.797357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b45abda9-3d5e-40d7-9742-f94ad8f4072f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049147 May 25 16:36:35.797933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 569/1155] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:35 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:36.810988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-25629db0-fd99-4442-86ff-e44d8f7e4d63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:36.812186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-25629db0-fd99-4442-86ff-e44d8f7e4d63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:36.820094 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-25629db0-fd99-4442-86ff-e44d8f7e4d63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:36.820644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-25629db0-fd99-4442-86ff-e44d8f7e4d63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:36.821545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-25629db0-fd99-4442-86ff-e44d8f7e4d63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:36.822099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-25629db0-fd99-4442-86ff-e44d8f7e4d63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:36.856038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-25629db0-fd99-4442-86ff-e44d8f7e4d63 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049777 May 25 16:36:36.856611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 587/1156] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:36 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:37.871769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-14d8fb95-6699-488c-9300-1f2806338e07 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:37.873284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-14d8fb95-6699-488c-9300-1f2806338e07 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:37.882698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-14d8fb95-6699-488c-9300-1f2806338e07 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:37.883125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-14d8fb95-6699-488c-9300-1f2806338e07 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:37.883947 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-14d8fb95-6699-488c-9300-1f2806338e07 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:37.884380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-14d8fb95-6699-488c-9300-1f2806338e07 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:37.914240 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-14d8fb95-6699-488c-9300-1f2806338e07 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049088 May 25 16:36:37.915348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 570/1157] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:37 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:38.926678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7fb8b2c7-9d9b-48e7-b7ed-3194ea873336 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:38.927530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7fb8b2c7-9d9b-48e7-b7ed-3194ea873336 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:38.934390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7fb8b2c7-9d9b-48e7-b7ed-3194ea873336 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:38.934857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7fb8b2c7-9d9b-48e7-b7ed-3194ea873336 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:38.935629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7fb8b2c7-9d9b-48e7-b7ed-3194ea873336 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:38.936025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7fb8b2c7-9d9b-48e7-b7ed-3194ea873336 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:38.966668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7fb8b2c7-9d9b-48e7-b7ed-3194ea873336 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044115 May 25 16:36:38.967213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 588/1158] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:38 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:39.982573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6aed2507-3a67-44f7-b57a-df732f29f26f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:39.983996 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6aed2507-3a67-44f7-b57a-df732f29f26f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:39.993283 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6aed2507-3a67-44f7-b57a-df732f29f26f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:39.993730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6aed2507-3a67-44f7-b57a-df732f29f26f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:39.994479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6aed2507-3a67-44f7-b57a-df732f29f26f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:39.994903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6aed2507-3a67-44f7-b57a-df732f29f26f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:40.024254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6aed2507-3a67-44f7-b57a-df732f29f26f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048371 May 25 16:36:40.024814 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 571/1159] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:39 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:41.040199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dfd87ba0-c52a-4803-93a2-e7dbbe5b0355 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:41.041333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dfd87ba0-c52a-4803-93a2-e7dbbe5b0355 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:41.050181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dfd87ba0-c52a-4803-93a2-e7dbbe5b0355 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:41.050777 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dfd87ba0-c52a-4803-93a2-e7dbbe5b0355 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:41.051802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dfd87ba0-c52a-4803-93a2-e7dbbe5b0355 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:41.052382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dfd87ba0-c52a-4803-93a2-e7dbbe5b0355 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:41.086524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dfd87ba0-c52a-4803-93a2-e7dbbe5b0355 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053121 May 25 16:36:41.090266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 589/1160] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:41 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:42.098293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-74f462b6-b93a-4090-b22a-6052ee4fad3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:42.099260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-74f462b6-b93a-4090-b22a-6052ee4fad3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:42.106265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-74f462b6-b93a-4090-b22a-6052ee4fad3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:42.106722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-74f462b6-b93a-4090-b22a-6052ee4fad3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:42.107512 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-74f462b6-b93a-4090-b22a-6052ee4fad3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:42.107951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-74f462b6-b93a-4090-b22a-6052ee4fad3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:42.138559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-74f462b6-b93a-4090-b22a-6052ee4fad3c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044237 May 25 16:36:42.142505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 572/1161] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:42 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:43.153899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-27a10002-4a07-4fd1-8ef8-9ec73664d359 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:43.155445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-27a10002-4a07-4fd1-8ef8-9ec73664d359 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:43.165599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27a10002-4a07-4fd1-8ef8-9ec73664d359 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:43.166045 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27a10002-4a07-4fd1-8ef8-9ec73664d359 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:43.166929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27a10002-4a07-4fd1-8ef8-9ec73664d359 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:43.167359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27a10002-4a07-4fd1-8ef8-9ec73664d359 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:43.197687 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-27a10002-4a07-4fd1-8ef8-9ec73664d359 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050434 May 25 16:36:43.198245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 590/1162] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:43 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:44.209833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cfda77a7-d470-41d6-bc09-d322e40809e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:44.210751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cfda77a7-d470-41d6-bc09-d322e40809e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:44.213128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:36:44.219682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfda77a7-d470-41d6-bc09-d322e40809e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:44.220159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfda77a7-d470-41d6-bc09-d322e40809e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:44.220935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfda77a7-d470-41d6-bc09-d322e40809e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:44.221368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfda77a7-d470-41d6-bc09-d322e40809e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:44.224115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=90 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:36:44.253449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cfda77a7-d470-41d6-bc09-d322e40809e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047677 May 25 16:36:44.254030 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 573/1163] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:44 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:45.269107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8c4f83f4-1d0e-4291-a745-b75637c76640 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:45.270592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8c4f83f4-1d0e-4291-a745-b75637c76640 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:45.273489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=30 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:36:45.279342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8c4f83f4-1d0e-4291-a745-b75637c76640 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:45.279757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8c4f83f4-1d0e-4291-a745-b75637c76640 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:45.280523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8c4f83f4-1d0e-4291-a745-b75637c76640 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:45.280952 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8c4f83f4-1d0e-4291-a745-b75637c76640 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:45.285054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=90 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:36:45.311498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8c4f83f4-1d0e-4291-a745-b75637c76640 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049208 May 25 16:36:45.312077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 591/1164] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:45 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:46.327454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c9ad490f-6390-47d4-9463-e6d9f07c7d5e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:46.329066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c9ad490f-6390-47d4-9463-e6d9f07c7d5e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:46.340160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9ad490f-6390-47d4-9463-e6d9f07c7d5e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:46.340894 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9ad490f-6390-47d4-9463-e6d9f07c7d5e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:46.342171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9ad490f-6390-47d4-9463-e6d9f07c7d5e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:46.342889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9ad490f-6390-47d4-9463-e6d9f07c7d5e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:46.375600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c9ad490f-6390-47d4-9463-e6d9f07c7d5e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054737 May 25 16:36:46.376525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 574/1165] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:46 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:47.389613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1ea6831e-e1dd-4fbc-9245-fe645a8dbe84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:47.390948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1ea6831e-e1dd-4fbc-9245-fe645a8dbe84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:47.398738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ea6831e-e1dd-4fbc-9245-fe645a8dbe84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:47.399233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ea6831e-e1dd-4fbc-9245-fe645a8dbe84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:47.400068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ea6831e-e1dd-4fbc-9245-fe645a8dbe84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:47.400414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ea6831e-e1dd-4fbc-9245-fe645a8dbe84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:47.430439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1ea6831e-e1dd-4fbc-9245-fe645a8dbe84 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047325 May 25 16:36:47.431007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 592/1166] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:47 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:48.441936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-413851d9-37aa-4423-8e9d-1f30f8709e3d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:48.442892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-413851d9-37aa-4423-8e9d-1f30f8709e3d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:48.449846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-413851d9-37aa-4423-8e9d-1f30f8709e3d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:48.450299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-413851d9-37aa-4423-8e9d-1f30f8709e3d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:48.451060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-413851d9-37aa-4423-8e9d-1f30f8709e3d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:48.451513 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-413851d9-37aa-4423-8e9d-1f30f8709e3d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:48.481570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-413851d9-37aa-4423-8e9d-1f30f8709e3d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043725 May 25 16:36:48.482355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 575/1167] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:48 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:49.497892 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8e223b2a-e024-4437-9c83-33b897634831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:49.499284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8e223b2a-e024-4437-9c83-33b897634831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:49.510070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e223b2a-e024-4437-9c83-33b897634831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:49.510487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e223b2a-e024-4437-9c83-33b897634831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:49.511782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e223b2a-e024-4437-9c83-33b897634831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:49.512529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e223b2a-e024-4437-9c83-33b897634831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:49.546134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8e223b2a-e024-4437-9c83-33b897634831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055168 May 25 16:36:49.546703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 593/1168] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:49 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:50.562135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-11e8c645-877d-48e4-a9cc-59da16d28ec4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:50.563667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-11e8c645-877d-48e4-a9cc-59da16d28ec4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:50.573365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11e8c645-877d-48e4-a9cc-59da16d28ec4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:50.573818 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11e8c645-877d-48e4-a9cc-59da16d28ec4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:50.574581 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11e8c645-877d-48e4-a9cc-59da16d28ec4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:50.575013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11e8c645-877d-48e4-a9cc-59da16d28ec4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:50.605084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-11e8c645-877d-48e4-a9cc-59da16d28ec4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049588 May 25 16:36:50.605574 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 576/1169] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:50 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:51.621117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-455d5311-4d12-408e-a584-74c9bb5c9ea0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:51.622531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-455d5311-4d12-408e-a584-74c9bb5c9ea0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:51.633029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-455d5311-4d12-408e-a584-74c9bb5c9ea0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:51.633517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-455d5311-4d12-408e-a584-74c9bb5c9ea0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:51.634231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-455d5311-4d12-408e-a584-74c9bb5c9ea0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:51.634660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-455d5311-4d12-408e-a584-74c9bb5c9ea0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:51.664315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-455d5311-4d12-408e-a584-74c9bb5c9ea0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050007 May 25 16:36:51.665211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 594/1170] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:51 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:52.679985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-56c0dbc6-1c08-49b1-b63c-20018b86adcb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:52.681998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-56c0dbc6-1c08-49b1-b63c-20018b86adcb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:52.688602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56c0dbc6-1c08-49b1-b63c-20018b86adcb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:52.689044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56c0dbc6-1c08-49b1-b63c-20018b86adcb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:52.689995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56c0dbc6-1c08-49b1-b63c-20018b86adcb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:52.690426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56c0dbc6-1c08-49b1-b63c-20018b86adcb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:52.719765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-56c0dbc6-1c08-49b1-b63c-20018b86adcb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045898 May 25 16:36:52.720527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 577/1171] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:52 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:53.735938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-13fbc548-aaf6-4a16-963c-4603859c1bab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:53.737466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-13fbc548-aaf6-4a16-963c-4603859c1bab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:53.747729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13fbc548-aaf6-4a16-963c-4603859c1bab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:53.748483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13fbc548-aaf6-4a16-963c-4603859c1bab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:53.749739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13fbc548-aaf6-4a16-963c-4603859c1bab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:53.750461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13fbc548-aaf6-4a16-963c-4603859c1bab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:53.782414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-13fbc548-aaf6-4a16-963c-4603859c1bab tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053248 May 25 16:36:53.782980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 595/1172] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:53 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:54.798645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3d13fc37-1892-4472-b419-eaba369ef1d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:54.800158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3d13fc37-1892-4472-b419-eaba369ef1d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:54.810069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d13fc37-1892-4472-b419-eaba369ef1d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:54.810517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d13fc37-1892-4472-b419-eaba369ef1d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:54.811321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d13fc37-1892-4472-b419-eaba369ef1d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:54.812008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d13fc37-1892-4472-b419-eaba369ef1d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:54.841191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3d13fc37-1892-4472-b419-eaba369ef1d3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049143 May 25 16:36:54.841757 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 578/1173] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:54 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:55.855136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4d0fcfcc-eb02-4259-a112-e91f0bb82644 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:55.856477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4d0fcfcc-eb02-4259-a112-e91f0bb82644 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:55.865139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d0fcfcc-eb02-4259-a112-e91f0bb82644 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:55.865785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d0fcfcc-eb02-4259-a112-e91f0bb82644 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:55.866925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d0fcfcc-eb02-4259-a112-e91f0bb82644 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:55.867516 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d0fcfcc-eb02-4259-a112-e91f0bb82644 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:55.900540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4d0fcfcc-eb02-4259-a112-e91f0bb82644 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050573 May 25 16:36:55.901119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 596/1174] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:55 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:56.917215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1bd7fe8e-183a-4969-89c1-b2079cb72d71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:56.918686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1bd7fe8e-183a-4969-89c1-b2079cb72d71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:56.929405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1bd7fe8e-183a-4969-89c1-b2079cb72d71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:56.929849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1bd7fe8e-183a-4969-89c1-b2079cb72d71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:56.930617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1bd7fe8e-183a-4969-89c1-b2079cb72d71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:56.931056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1bd7fe8e-183a-4969-89c1-b2079cb72d71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:56.961184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1bd7fe8e-183a-4969-89c1-b2079cb72d71 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051325 May 25 16:36:56.961765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 579/1175] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:56 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:57.977300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-893fc1c1-c6a5-4be1-84b9-c6503e980145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:57.978792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-893fc1c1-c6a5-4be1-84b9-c6503e980145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:57.989050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-893fc1c1-c6a5-4be1-84b9-c6503e980145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:57.989802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-893fc1c1-c6a5-4be1-84b9-c6503e980145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:57.991206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-893fc1c1-c6a5-4be1-84b9-c6503e980145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:57.991646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-893fc1c1-c6a5-4be1-84b9-c6503e980145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:58.022930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-893fc1c1-c6a5-4be1-84b9-c6503e980145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052236 May 25 16:36:58.023573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 597/1176] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:57 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:36:59.038422 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b3c9f011-1f3c-49af-a3ab-8a86807fa366 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:36:59.039398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b3c9f011-1f3c-49af-a3ab-8a86807fa366 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:36:59.046184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3c9f011-1f3c-49af-a3ab-8a86807fa366 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:59.046818 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3c9f011-1f3c-49af-a3ab-8a86807fa366 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:59.048999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3c9f011-1f3c-49af-a3ab-8a86807fa366 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:36:59.048999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3c9f011-1f3c-49af-a3ab-8a86807fa366 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:36:59.081288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b3c9f011-1f3c-49af-a3ab-8a86807fa366 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049622 May 25 16:36:59.081882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 580/1177] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:36:59 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:00.093300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-573dd14c-2c8b-4b6b-b32b-c1bb4c909844 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:00.094217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-573dd14c-2c8b-4b6b-b32b-c1bb4c909844 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:00.101101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-573dd14c-2c8b-4b6b-b32b-c1bb4c909844 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:00.101545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-573dd14c-2c8b-4b6b-b32b-c1bb4c909844 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:00.102292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-573dd14c-2c8b-4b6b-b32b-c1bb4c909844 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:00.102719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-573dd14c-2c8b-4b6b-b32b-c1bb4c909844 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:00.132199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-573dd14c-2c8b-4b6b-b32b-c1bb4c909844 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042947 May 25 16:37:00.132767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 598/1178] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:00 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:01.148406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c9895080-fb2c-4b03-9f82-e662c915d01e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:01.149902 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c9895080-fb2c-4b03-9f82-e662c915d01e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:01.160768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9895080-fb2c-4b03-9f82-e662c915d01e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:01.161536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9895080-fb2c-4b03-9f82-e662c915d01e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:01.162797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9895080-fb2c-4b03-9f82-e662c915d01e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:01.163518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9895080-fb2c-4b03-9f82-e662c915d01e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:01.195528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c9895080-fb2c-4b03-9f82-e662c915d01e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053952 May 25 16:37:01.196308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 581/1179] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:01 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:02.211173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2d90d1c7-8cf9-454b-9500-01b79e4e5037 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:02.212242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2d90d1c7-8cf9-454b-9500-01b79e4e5037 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:02.219945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d90d1c7-8cf9-454b-9500-01b79e4e5037 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:02.220442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d90d1c7-8cf9-454b-9500-01b79e4e5037 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:02.221675 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d90d1c7-8cf9-454b-9500-01b79e4e5037 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:02.222143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d90d1c7-8cf9-454b-9500-01b79e4e5037 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:02.258415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2d90d1c7-8cf9-454b-9500-01b79e4e5037 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053595 May 25 16:37:02.258979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 599/1180] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:02 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:03.273733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cad4a2ff-9679-4a1b-9f38-becc82f31d49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:03.274780 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cad4a2ff-9679-4a1b-9f38-becc82f31d49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:03.282419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cad4a2ff-9679-4a1b-9f38-becc82f31d49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:03.282907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cad4a2ff-9679-4a1b-9f38-becc82f31d49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:03.283825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cad4a2ff-9679-4a1b-9f38-becc82f31d49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:03.284336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cad4a2ff-9679-4a1b-9f38-becc82f31d49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:03.315193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cad4a2ff-9679-4a1b-9f38-becc82f31d49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047782 May 25 16:37:03.316125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 582/1181] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:03 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:04.327249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-428f0cab-02d6-40de-b42c-182a9941a035 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:04.328184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-428f0cab-02d6-40de-b42c-182a9941a035 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:04.334864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-428f0cab-02d6-40de-b42c-182a9941a035 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:04.335352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-428f0cab-02d6-40de-b42c-182a9941a035 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:04.336123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-428f0cab-02d6-40de-b42c-182a9941a035 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:04.336570 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-428f0cab-02d6-40de-b42c-182a9941a035 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:04.366861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-428f0cab-02d6-40de-b42c-182a9941a035 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043486 May 25 16:37:04.367470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 600/1182] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:04 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:05.379746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b4beb8ba-e05a-424c-85c9-9c3c98643d6b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:05.380734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b4beb8ba-e05a-424c-85c9-9c3c98643d6b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:05.389094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4beb8ba-e05a-424c-85c9-9c3c98643d6b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:05.389633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4beb8ba-e05a-424c-85c9-9c3c98643d6b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:05.390487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4beb8ba-e05a-424c-85c9-9c3c98643d6b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:05.391023 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4beb8ba-e05a-424c-85c9-9c3c98643d6b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:05.421076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b4beb8ba-e05a-424c-85c9-9c3c98643d6b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046155 May 25 16:37:05.421826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 583/1183] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:05 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:06.433034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8abdbd79-b1c4-4978-a4c9-0da6423f294b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:06.433960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8abdbd79-b1c4-4978-a4c9-0da6423f294b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:06.440819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8abdbd79-b1c4-4978-a4c9-0da6423f294b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:06.441279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8abdbd79-b1c4-4978-a4c9-0da6423f294b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:06.442050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8abdbd79-b1c4-4978-a4c9-0da6423f294b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:06.442489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8abdbd79-b1c4-4978-a4c9-0da6423f294b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:06.472597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8abdbd79-b1c4-4978-a4c9-0da6423f294b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043431 May 25 16:37:06.473169 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 601/1184] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:06 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:07.484436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0790212f-a473-4834-97b6-ef22be619af9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:07.485478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0790212f-a473-4834-97b6-ef22be619af9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:07.492516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0790212f-a473-4834-97b6-ef22be619af9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:07.493081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0790212f-a473-4834-97b6-ef22be619af9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:07.493918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0790212f-a473-4834-97b6-ef22be619af9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:07.494420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0790212f-a473-4834-97b6-ef22be619af9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:07.524886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0790212f-a473-4834-97b6-ef22be619af9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044407 May 25 16:37:07.525456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 584/1185] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:07 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:08.536470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1edc4e49-60df-405b-be10-0e71e5530540 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:08.537405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1edc4e49-60df-405b-be10-0e71e5530540 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:08.545286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1edc4e49-60df-405b-be10-0e71e5530540 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:08.545739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1edc4e49-60df-405b-be10-0e71e5530540 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:08.546537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1edc4e49-60df-405b-be10-0e71e5530540 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:08.546967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1edc4e49-60df-405b-be10-0e71e5530540 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:08.576501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1edc4e49-60df-405b-be10-0e71e5530540 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043859 May 25 16:37:08.577071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 602/1186] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:08 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:09.589355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-85623e0e-5b49-42ec-84b2-d7d745385d11 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:09.589355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-85623e0e-5b49-42ec-84b2-d7d745385d11 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:09.598888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85623e0e-5b49-42ec-84b2-d7d745385d11 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:09.599666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85623e0e-5b49-42ec-84b2-d7d745385d11 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:09.600740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85623e0e-5b49-42ec-84b2-d7d745385d11 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:09.601152 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85623e0e-5b49-42ec-84b2-d7d745385d11 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:09.631610 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-85623e0e-5b49-42ec-84b2-d7d745385d11 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046863 May 25 16:37:09.632205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 585/1187] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:09 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:10.644128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f7f2e64e-6cf9-494f-9364-01e175fa4e52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:10.645062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f7f2e64e-6cf9-494f-9364-01e175fa4e52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:10.651737 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f7f2e64e-6cf9-494f-9364-01e175fa4e52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:10.652168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f7f2e64e-6cf9-494f-9364-01e175fa4e52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:10.652935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f7f2e64e-6cf9-494f-9364-01e175fa4e52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:10.653375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f7f2e64e-6cf9-494f-9364-01e175fa4e52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:10.685166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f7f2e64e-6cf9-494f-9364-01e175fa4e52 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044982 May 25 16:37:10.685760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 603/1188] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:10 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:11.697089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-51fd7e82-bfd0-4587-8723-48d0320b7831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:11.698198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-51fd7e82-bfd0-4587-8723-48d0320b7831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:11.704940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51fd7e82-bfd0-4587-8723-48d0320b7831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:11.705398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51fd7e82-bfd0-4587-8723-48d0320b7831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:11.706167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51fd7e82-bfd0-4587-8723-48d0320b7831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:11.706604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51fd7e82-bfd0-4587-8723-48d0320b7831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:11.737009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-51fd7e82-bfd0-4587-8723-48d0320b7831 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044130 May 25 16:37:11.740752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 586/1189] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:11 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:12.748008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c69f6b14-172c-4205-a018-0d0dd5263f49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:12.748985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c69f6b14-172c-4205-a018-0d0dd5263f49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:12.755635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c69f6b14-172c-4205-a018-0d0dd5263f49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:12.756078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c69f6b14-172c-4205-a018-0d0dd5263f49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:12.756845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c69f6b14-172c-4205-a018-0d0dd5263f49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:12.757282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c69f6b14-172c-4205-a018-0d0dd5263f49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:12.786721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c69f6b14-172c-4205-a018-0d0dd5263f49 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042747 May 25 16:37:12.789368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 604/1190] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:12 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:13.798992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4fbb58e6-d8a7-42f7-9568-0907d311a95c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:13.800675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4fbb58e6-d8a7-42f7-9568-0907d311a95c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:13.808643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fbb58e6-d8a7-42f7-9568-0907d311a95c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:13.809099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fbb58e6-d8a7-42f7-9568-0907d311a95c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:13.809969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fbb58e6-d8a7-42f7-9568-0907d311a95c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:13.810470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4fbb58e6-d8a7-42f7-9568-0907d311a95c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:13.841136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4fbb58e6-d8a7-42f7-9568-0907d311a95c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047012 May 25 16:37:13.841700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 587/1191] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:13 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:14.852841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fd2b99e8-df20-4045-9840-9bc3b94a4522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:14.853841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fd2b99e8-df20-4045-9840-9bc3b94a4522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:14.860732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd2b99e8-df20-4045-9840-9bc3b94a4522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:14.861187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd2b99e8-df20-4045-9840-9bc3b94a4522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:14.861948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd2b99e8-df20-4045-9840-9bc3b94a4522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:14.862380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd2b99e8-df20-4045-9840-9bc3b94a4522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:14.893054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fd2b99e8-df20-4045-9840-9bc3b94a4522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044450 May 25 16:37:14.893630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 605/1192] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:14 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:15.906303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9557e9d8-69a8-41d6-b93f-87e70e607ed2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:15.908332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9557e9d8-69a8-41d6-b93f-87e70e607ed2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:15.910654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:37:15.916511 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9557e9d8-69a8-41d6-b93f-87e70e607ed2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:15.916962 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9557e9d8-69a8-41d6-b93f-87e70e607ed2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:15.917695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9557e9d8-69a8-41d6-b93f-87e70e607ed2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:15.918148 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9557e9d8-69a8-41d6-b93f-87e70e607ed2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:15.920827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=90 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:37:15.950542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9557e9d8-69a8-41d6-b93f-87e70e607ed2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048595 May 25 16:37:15.951183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 588/1193] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:15 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:16.962187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8715c580-20d9-44c9-aa2a-45a4e2c1d09d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:16.963144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8715c580-20d9-44c9-aa2a-45a4e2c1d09d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:16.965587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=30 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:37:16.971018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8715c580-20d9-44c9-aa2a-45a4e2c1d09d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:16.971497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8715c580-20d9-44c9-aa2a-45a4e2c1d09d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:16.972261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8715c580-20d9-44c9-aa2a-45a4e2c1d09d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:16.972917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8715c580-20d9-44c9-aa2a-45a4e2c1d09d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:16.976878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=90 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:37:17.003296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8715c580-20d9-44c9-aa2a-45a4e2c1d09d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045026 May 25 16:37:17.003835 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 606/1194] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:16 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:18.014053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a9a875d3-9ae8-4cd9-bbc3-e0c3efb14639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:18.014970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a9a875d3-9ae8-4cd9-bbc3-e0c3efb14639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:18.021770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9a875d3-9ae8-4cd9-bbc3-e0c3efb14639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:18.022235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9a875d3-9ae8-4cd9-bbc3-e0c3efb14639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:18.022990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9a875d3-9ae8-4cd9-bbc3-e0c3efb14639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:18.023455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9a875d3-9ae8-4cd9-bbc3-e0c3efb14639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:18.054655 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a9a875d3-9ae8-4cd9-bbc3-e0c3efb14639 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044556 May 25 16:37:18.055251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 589/1195] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:18 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:19.069972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-db1ce05a-a92f-48b7-a4ec-a6dc7a3178e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:19.069972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-db1ce05a-a92f-48b7-a4ec-a6dc7a3178e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:19.074166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db1ce05a-a92f-48b7-a4ec-a6dc7a3178e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:19.074627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db1ce05a-a92f-48b7-a4ec-a6dc7a3178e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:19.075565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db1ce05a-a92f-48b7-a4ec-a6dc7a3178e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:19.076004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db1ce05a-a92f-48b7-a4ec-a6dc7a3178e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:19.107182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-db1ce05a-a92f-48b7-a4ec-a6dc7a3178e1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045004 May 25 16:37:19.107729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 607/1196] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:19 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:20.119635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-990e8018-3b5e-4678-955e-349e6bd12439 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:20.120584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-990e8018-3b5e-4678-955e-349e6bd12439 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:20.128324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-990e8018-3b5e-4678-955e-349e6bd12439 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:20.128778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-990e8018-3b5e-4678-955e-349e6bd12439 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:20.129527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-990e8018-3b5e-4678-955e-349e6bd12439 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:20.129986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-990e8018-3b5e-4678-955e-349e6bd12439 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:20.161416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-990e8018-3b5e-4678-955e-349e6bd12439 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045856 May 25 16:37:20.162319 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 590/1197] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:20 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:21.175072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d41911f6-4b0c-4e0e-8728-9bddad88803c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:21.176261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d41911f6-4b0c-4e0e-8728-9bddad88803c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:21.183537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d41911f6-4b0c-4e0e-8728-9bddad88803c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:21.183991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d41911f6-4b0c-4e0e-8728-9bddad88803c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:21.184752 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d41911f6-4b0c-4e0e-8728-9bddad88803c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:21.185186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d41911f6-4b0c-4e0e-8728-9bddad88803c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:21.214836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d41911f6-4b0c-4e0e-8728-9bddad88803c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044776 May 25 16:37:21.215444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 608/1198] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:21 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:22.226248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8638d59f-21b6-46a4-affe-133e715bdd2a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:22.227249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8638d59f-21b6-46a4-affe-133e715bdd2a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:22.233963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8638d59f-21b6-46a4-affe-133e715bdd2a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:22.234406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8638d59f-21b6-46a4-affe-133e715bdd2a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:22.235391 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8638d59f-21b6-46a4-affe-133e715bdd2a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:22.235678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8638d59f-21b6-46a4-affe-133e715bdd2a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:22.278934 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8638d59f-21b6-46a4-affe-133e715bdd2a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056643 May 25 16:37:22.279566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 591/1199] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:22 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:23.297695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4a007788-b47f-4ddf-9745-0188a660d34e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:23.298967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4a007788-b47f-4ddf-9745-0188a660d34e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:23.306529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a007788-b47f-4ddf-9745-0188a660d34e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:23.306987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a007788-b47f-4ddf-9745-0188a660d34e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:23.307787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a007788-b47f-4ddf-9745-0188a660d34e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:23.308224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a007788-b47f-4ddf-9745-0188a660d34e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:23.323304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:37:23.323492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:37:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments (ip 10.222.0.45) !!! May 25 16:37:23.345853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4a007788-b47f-4ddf-9745-0188a660d34e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053434 May 25 16:37:23.346455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 609/1200] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:23 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:24.357910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fec75a78-5348-44c0-b815-e83be30616cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:24.358842 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fec75a78-5348-44c0-b815-e83be30616cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:24.365721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec75a78-5348-44c0-b815-e83be30616cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:24.366155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec75a78-5348-44c0-b815-e83be30616cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:24.366921 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec75a78-5348-44c0-b815-e83be30616cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:24.367399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec75a78-5348-44c0-b815-e83be30616cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:24.406994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fec75a78-5348-44c0-b815-e83be30616cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053104 May 25 16:37:24.407791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 592/1201] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:24 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:25.420263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b22ed1d0-6af3-41fd-a72c-5d2be250f16e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:25.421377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b22ed1d0-6af3-41fd-a72c-5d2be250f16e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:25.428621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b22ed1d0-6af3-41fd-a72c-5d2be250f16e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:25.428621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b22ed1d0-6af3-41fd-a72c-5d2be250f16e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:25.429309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b22ed1d0-6af3-41fd-a72c-5d2be250f16e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:25.429750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b22ed1d0-6af3-41fd-a72c-5d2be250f16e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:25.461827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b22ed1d0-6af3-41fd-a72c-5d2be250f16e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047182 May 25 16:37:25.462563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 610/1202] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:25 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:26.478179 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-82284cbf-78f3-476d-ac5b-f3840efde79d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:26.479852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-82284cbf-78f3-476d-ac5b-f3840efde79d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:26.490107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82284cbf-78f3-476d-ac5b-f3840efde79d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:26.490838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82284cbf-78f3-476d-ac5b-f3840efde79d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:26.491645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82284cbf-78f3-476d-ac5b-f3840efde79d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:26.492063 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82284cbf-78f3-476d-ac5b-f3840efde79d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:26.522511 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-82284cbf-78f3-476d-ac5b-f3840efde79d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051972 May 25 16:37:26.526982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 593/1203] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:26 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:27.534469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e4763687-2f5f-4843-925c-9205a22ce8a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:27.535447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e4763687-2f5f-4843-925c-9205a22ce8a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:27.542723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4763687-2f5f-4843-925c-9205a22ce8a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:27.543145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4763687-2f5f-4843-925c-9205a22ce8a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:27.543958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4763687-2f5f-4843-925c-9205a22ce8a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:27.544387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4763687-2f5f-4843-925c-9205a22ce8a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:27.574230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e4763687-2f5f-4843-925c-9205a22ce8a2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043988 May 25 16:37:27.574932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 611/1204] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:27 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:28.590631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-62e6e571-3249-4ed9-8a86-de432fb99bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:28.592243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-62e6e571-3249-4ed9-8a86-de432fb99bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:28.602993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62e6e571-3249-4ed9-8a86-de432fb99bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:28.603801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62e6e571-3249-4ed9-8a86-de432fb99bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:28.605067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62e6e571-3249-4ed9-8a86-de432fb99bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:28.605789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62e6e571-3249-4ed9-8a86-de432fb99bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:28.640610 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-62e6e571-3249-4ed9-8a86-de432fb99bb1 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056740 May 25 16:37:28.641369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 594/1205] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:28 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:29.654553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9f88b108-880b-4907-8f09-1997e9cdc891 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:29.655797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9f88b108-880b-4907-8f09-1997e9cdc891 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:29.664548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f88b108-880b-4907-8f09-1997e9cdc891 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:29.665311 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f88b108-880b-4907-8f09-1997e9cdc891 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:29.666072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f88b108-880b-4907-8f09-1997e9cdc891 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:29.666502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f88b108-880b-4907-8f09-1997e9cdc891 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:29.699181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9f88b108-880b-4907-8f09-1997e9cdc891 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050398 May 25 16:37:29.699942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 612/1206] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:29 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:30.715137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-18044d68-8da6-42fc-8130-c526a277cd73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:30.716796 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-18044d68-8da6-42fc-8130-c526a277cd73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:30.728318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-18044d68-8da6-42fc-8130-c526a277cd73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:30.729067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-18044d68-8da6-42fc-8130-c526a277cd73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:30.730324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-18044d68-8da6-42fc-8130-c526a277cd73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:30.731036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-18044d68-8da6-42fc-8130-c526a277cd73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:30.765311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-18044d68-8da6-42fc-8130-c526a277cd73 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056837 May 25 16:37:30.765888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 595/1207] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:30 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:31.781476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e3f19312-5bc9-4a3b-902b-645d8104676a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:31.782979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e3f19312-5bc9-4a3b-902b-645d8104676a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:31.791955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3f19312-5bc9-4a3b-902b-645d8104676a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:31.792305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3f19312-5bc9-4a3b-902b-645d8104676a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:31.793076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3f19312-5bc9-4a3b-902b-645d8104676a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:31.793512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e3f19312-5bc9-4a3b-902b-645d8104676a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:31.826679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e3f19312-5bc9-4a3b-902b-645d8104676a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051913 May 25 16:37:31.827349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 613/1208] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:31 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:32.843041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4b58949f-e66e-49c8-a909-aefdf43f5c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:32.844595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4b58949f-e66e-49c8-a909-aefdf43f5c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:32.853721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4b58949f-e66e-49c8-a909-aefdf43f5c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:32.854174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4b58949f-e66e-49c8-a909-aefdf43f5c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:32.854967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4b58949f-e66e-49c8-a909-aefdf43f5c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:32.855459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4b58949f-e66e-49c8-a909-aefdf43f5c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:32.885458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4b58949f-e66e-49c8-a909-aefdf43f5c04 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049651 May 25 16:37:32.886675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 596/1209] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:32 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:33.898666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ac41095c-8eb1-4fb4-8d81-9ec7096a59e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:33.899644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ac41095c-8eb1-4fb4-8d81-9ec7096a59e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:33.906674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac41095c-8eb1-4fb4-8d81-9ec7096a59e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:33.907119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac41095c-8eb1-4fb4-8d81-9ec7096a59e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:33.908043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac41095c-8eb1-4fb4-8d81-9ec7096a59e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:33.908485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac41095c-8eb1-4fb4-8d81-9ec7096a59e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:33.939849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ac41095c-8eb1-4fb4-8d81-9ec7096a59e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045734 May 25 16:37:33.940734 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 614/1210] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:33 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:34.956075 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d77b7898-3484-46bf-b51d-5141c90c5147 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:34.957570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d77b7898-3484-46bf-b51d-5141c90c5147 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:34.968704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d77b7898-3484-46bf-b51d-5141c90c5147 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:34.969621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d77b7898-3484-46bf-b51d-5141c90c5147 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:34.970892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d77b7898-3484-46bf-b51d-5141c90c5147 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:34.971658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d77b7898-3484-46bf-b51d-5141c90c5147 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:35.004961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d77b7898-3484-46bf-b51d-5141c90c5147 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055620 May 25 16:37:35.005514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 597/1211] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:34 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:36.021750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-64700e16-e101-4ed6-8136-a3304a488b5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:36.022722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-64700e16-e101-4ed6-8136-a3304a488b5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:36.030081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64700e16-e101-4ed6-8136-a3304a488b5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:36.030558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64700e16-e101-4ed6-8136-a3304a488b5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:36.031398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64700e16-e101-4ed6-8136-a3304a488b5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:36.031857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64700e16-e101-4ed6-8136-a3304a488b5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:36.063067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-64700e16-e101-4ed6-8136-a3304a488b5d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048231 May 25 16:37:36.063672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 615/1212] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:36 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:37.074540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-60cdc0b3-aac6-4a40-af81-27046bcad9f6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:37.075526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-60cdc0b3-aac6-4a40-af81-27046bcad9f6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:37.082576 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60cdc0b3-aac6-4a40-af81-27046bcad9f6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:37.083067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60cdc0b3-aac6-4a40-af81-27046bcad9f6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:37.083932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60cdc0b3-aac6-4a40-af81-27046bcad9f6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:37.084410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60cdc0b3-aac6-4a40-af81-27046bcad9f6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:37.114466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-60cdc0b3-aac6-4a40-af81-27046bcad9f6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044127 May 25 16:37:37.115263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 598/1213] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:37 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:38.131072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ed024d92-973c-45be-9609-c943b487635b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:38.132827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ed024d92-973c-45be-9609-c943b487635b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:38.144998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed024d92-973c-45be-9609-c943b487635b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:38.145855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed024d92-973c-45be-9609-c943b487635b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:38.147326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed024d92-973c-45be-9609-c943b487635b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:38.148041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ed024d92-973c-45be-9609-c943b487635b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:38.179410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ed024d92-973c-45be-9609-c943b487635b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055719 May 25 16:37:38.183342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 616/1214] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:38 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:38.782392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:37:38.783285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:37:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:37:39.194379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-168df1d7-1688-4fa3-a332-faf351798e22 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:39.195949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-168df1d7-1688-4fa3-a332-faf351798e22 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:39.206709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-168df1d7-1688-4fa3-a332-faf351798e22 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:39.207595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-168df1d7-1688-4fa3-a332-faf351798e22 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:39.208891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-168df1d7-1688-4fa3-a332-faf351798e22 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:39.209625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-168df1d7-1688-4fa3-a332-faf351798e22 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:39.242849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-168df1d7-1688-4fa3-a332-faf351798e22 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055263 May 25 16:37:39.243677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 599/1215] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:39 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:40.255796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e5f18323-74c8-4316-bd6c-75cd0a2261d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:40.256938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e5f18323-74c8-4316-bd6c-75cd0a2261d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:40.263874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5f18323-74c8-4316-bd6c-75cd0a2261d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:40.264322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5f18323-74c8-4316-bd6c-75cd0a2261d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:40.265105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5f18323-74c8-4316-bd6c-75cd0a2261d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:40.265544 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5f18323-74c8-4316-bd6c-75cd0a2261d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:40.299816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e5f18323-74c8-4316-bd6c-75cd0a2261d9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048353 May 25 16:37:40.300371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 617/1216] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:40 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:41.315982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-30148b7c-7adf-44b6-ba09-8639b24308a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:41.317479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-30148b7c-7adf-44b6-ba09-8639b24308a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:41.328741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30148b7c-7adf-44b6-ba09-8639b24308a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:41.329645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30148b7c-7adf-44b6-ba09-8639b24308a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:41.330944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30148b7c-7adf-44b6-ba09-8639b24308a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:41.331822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30148b7c-7adf-44b6-ba09-8639b24308a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:41.364371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-30148b7c-7adf-44b6-ba09-8639b24308a9 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055084 May 25 16:37:41.364948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 600/1217] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:41 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:42.379226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bb096d82-04c3-45b2-9dd0-7065bbfc36bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:42.380524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bb096d82-04c3-45b2-9dd0-7065bbfc36bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:42.387799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bb096d82-04c3-45b2-9dd0-7065bbfc36bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:42.388251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bb096d82-04c3-45b2-9dd0-7065bbfc36bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:42.389014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bb096d82-04c3-45b2-9dd0-7065bbfc36bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:42.389450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bb096d82-04c3-45b2-9dd0-7065bbfc36bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:42.425956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bb096d82-04c3-45b2-9dd0-7065bbfc36bf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052671 May 25 16:37:42.426570 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 618/1218] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:42 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:43.433149 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:37:43.434071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:37:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:37:43.439941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-861ba250-b05a-4095-9f91-6af83535f458 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:43.441229 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-861ba250-b05a-4095-9f91-6af83535f458 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:43.450439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-861ba250-b05a-4095-9f91-6af83535f458 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:43.451105 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-861ba250-b05a-4095-9f91-6af83535f458 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:43.452292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-861ba250-b05a-4095-9f91-6af83535f458 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:43.452796 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-861ba250-b05a-4095-9f91-6af83535f458 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:43.484094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-861ba250-b05a-4095-9f91-6af83535f458 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049320 May 25 16:37:43.487928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 601/1219] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:43 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:44.496535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-022966e2-606a-44e6-a6f4-e1ba83ac0cdb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:44.498061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-022966e2-606a-44e6-a6f4-e1ba83ac0cdb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:44.504665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-022966e2-606a-44e6-a6f4-e1ba83ac0cdb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:44.505107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-022966e2-606a-44e6-a6f4-e1ba83ac0cdb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:44.505838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-022966e2-606a-44e6-a6f4-e1ba83ac0cdb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:44.506240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-022966e2-606a-44e6-a6f4-e1ba83ac0cdb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:44.540313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-022966e2-606a-44e6-a6f4-e1ba83ac0cdb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047354 May 25 16:37:44.540882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 619/1220] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:44 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:37:45.556157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d6fd8667-03f3-44ca-9398-a8774c9ff145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:45.557677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d6fd8667-03f3-44ca-9398-a8774c9ff145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:45.568786 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6fd8667-03f3-44ca-9398-a8774c9ff145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:45.569653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6fd8667-03f3-44ca-9398-a8774c9ff145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:45.570417 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6fd8667-03f3-44ca-9398-a8774c9ff145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:45.570854 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6fd8667-03f3-44ca-9398-a8774c9ff145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:45.603760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d6fd8667-03f3-44ca-9398-a8774c9ff145 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054279 May 25 16:37:45.604469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 602/1221] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:45 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:46.619111 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-055a7b15-08b4-4b2f-bed3-a6053887975e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:46.620378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-055a7b15-08b4-4b2f-bed3-a6053887975e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:46.629758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-055a7b15-08b4-4b2f-bed3-a6053887975e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:46.629758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-055a7b15-08b4-4b2f-bed3-a6053887975e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:46.630729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-055a7b15-08b4-4b2f-bed3-a6053887975e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:46.631355 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-055a7b15-08b4-4b2f-bed3-a6053887975e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:46.665484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-055a7b15-08b4-4b2f-bed3-a6053887975e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052090 May 25 16:37:46.666186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 620/1222] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:46 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:47.683031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-60d37941-52ba-438d-a738-81c4a5d37f21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:47.684621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-60d37941-52ba-438d-a738-81c4a5d37f21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:47.688855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:37:47.697488 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60d37941-52ba-438d-a738-81c4a5d37f21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:47.697943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60d37941-52ba-438d-a738-81c4a5d37f21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:47.698718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60d37941-52ba-438d-a738-81c4a5d37f21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:47.699187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60d37941-52ba-438d-a738-81c4a5d37f21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:47.701959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=90 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:37:47.730834 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-60d37941-52ba-438d-a738-81c4a5d37f21 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055913 May 25 16:37:47.731443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 603/1223] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:47 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:48.745263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b88a425a-ec14-46cb-8634-031a44e568ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:48.746275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b88a425a-ec14-46cb-8634-031a44e568ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:48.750937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=30 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:37:48.755997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b88a425a-ec14-46cb-8634-031a44e568ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:48.756550 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b88a425a-ec14-46cb-8634-031a44e568ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:48.757481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b88a425a-ec14-46cb-8634-031a44e568ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:48.758014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b88a425a-ec14-46cb-8634-031a44e568ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:48.762718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=90 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:37:48.791467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b88a425a-ec14-46cb-8634-031a44e568ce tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051793 May 25 16:37:48.792036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 621/1224] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:48 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:49.807827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3444eb59-e832-45af-ad7c-54ed461eb126 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:49.809354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3444eb59-e832-45af-ad7c-54ed461eb126 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:49.820439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3444eb59-e832-45af-ad7c-54ed461eb126 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:49.820886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3444eb59-e832-45af-ad7c-54ed461eb126 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:49.821637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3444eb59-e832-45af-ad7c-54ed461eb126 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:49.822066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3444eb59-e832-45af-ad7c-54ed461eb126 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:49.852366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3444eb59-e832-45af-ad7c-54ed461eb126 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051237 May 25 16:37:49.852944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 604/1225] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:49 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:50.867278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9325874c-4507-4a46-80f0-da85139f6428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:50.868207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9325874c-4507-4a46-80f0-da85139f6428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:50.875077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9325874c-4507-4a46-80f0-da85139f6428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:50.875563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9325874c-4507-4a46-80f0-da85139f6428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:50.876325 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9325874c-4507-4a46-80f0-da85139f6428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:50.876758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9325874c-4507-4a46-80f0-da85139f6428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:50.906519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9325874c-4507-4a46-80f0-da85139f6428 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045252 May 25 16:37:50.907079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 622/1226] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:50 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:51.921523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5f825900-e86c-43b0-aa6d-73ed3d4b3200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:51.923034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5f825900-e86c-43b0-aa6d-73ed3d4b3200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:51.931434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f825900-e86c-43b0-aa6d-73ed3d4b3200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:51.931847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f825900-e86c-43b0-aa6d-73ed3d4b3200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:51.932603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f825900-e86c-43b0-aa6d-73ed3d4b3200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:51.933028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f825900-e86c-43b0-aa6d-73ed3d4b3200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:51.966396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5f825900-e86c-43b0-aa6d-73ed3d4b3200 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050798 May 25 16:37:51.967822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 605/1227] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:51 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:52.981568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-363dc3c3-6133-429b-86c2-8d8067d4c48e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:52.982616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-363dc3c3-6133-429b-86c2-8d8067d4c48e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:52.990914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-363dc3c3-6133-429b-86c2-8d8067d4c48e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:52.991514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-363dc3c3-6133-429b-86c2-8d8067d4c48e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:52.992463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-363dc3c3-6133-429b-86c2-8d8067d4c48e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:52.993010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-363dc3c3-6133-429b-86c2-8d8067d4c48e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:53.026210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-363dc3c3-6133-429b-86c2-8d8067d4c48e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049774 May 25 16:37:53.026779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 623/1228] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:52 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:54.042706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b195e933-bc53-43f3-9dc7-43df7185895c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:54.044273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b195e933-bc53-43f3-9dc7-43df7185895c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:54.055256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b195e933-bc53-43f3-9dc7-43df7185895c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:54.055716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b195e933-bc53-43f3-9dc7-43df7185895c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:54.056501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b195e933-bc53-43f3-9dc7-43df7185895c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:54.056928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b195e933-bc53-43f3-9dc7-43df7185895c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:54.088978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b195e933-bc53-43f3-9dc7-43df7185895c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053403 May 25 16:37:54.089539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 606/1229] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:54 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:55.103465 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-eac5f182-dbbb-4b2b-afdf-84d621044ffd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:55.104394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-eac5f182-dbbb-4b2b-afdf-84d621044ffd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:55.112728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eac5f182-dbbb-4b2b-afdf-84d621044ffd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:55.113183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eac5f182-dbbb-4b2b-afdf-84d621044ffd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:55.113975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eac5f182-dbbb-4b2b-afdf-84d621044ffd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:55.114412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eac5f182-dbbb-4b2b-afdf-84d621044ffd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:55.144308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-eac5f182-dbbb-4b2b-afdf-84d621044ffd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046041 May 25 16:37:55.144922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 624/1230] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:55 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:56.163750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-14154c06-bde7-4bf6-bacc-12b84f093cb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:56.165397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-14154c06-bde7-4bf6-bacc-12b84f093cb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:56.174328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-14154c06-bde7-4bf6-bacc-12b84f093cb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:56.174924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-14154c06-bde7-4bf6-bacc-12b84f093cb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:56.176040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-14154c06-bde7-4bf6-bacc-12b84f093cb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:56.176551 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-14154c06-bde7-4bf6-bacc-12b84f093cb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:56.214992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-14154c06-bde7-4bf6-bacc-12b84f093cb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057875 May 25 16:37:56.215582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 607/1231] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:56 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:57.231115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-949d1dff-446d-487f-ba3f-c9bf20f4513c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:57.233961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-949d1dff-446d-487f-ba3f-c9bf20f4513c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:57.245443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-949d1dff-446d-487f-ba3f-c9bf20f4513c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:57.246243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-949d1dff-446d-487f-ba3f-c9bf20f4513c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:57.247556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-949d1dff-446d-487f-ba3f-c9bf20f4513c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:57.248281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-949d1dff-446d-487f-ba3f-c9bf20f4513c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:57.281211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-949d1dff-446d-487f-ba3f-c9bf20f4513c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056880 May 25 16:37:57.281790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 625/1232] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:57 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:58.298779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3cd22308-ecb8-4fbc-a778-7387d41e0445 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:58.300611 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3cd22308-ecb8-4fbc-a778-7387d41e0445 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:58.310258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3cd22308-ecb8-4fbc-a778-7387d41e0445 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:58.310719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3cd22308-ecb8-4fbc-a778-7387d41e0445 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:58.311526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3cd22308-ecb8-4fbc-a778-7387d41e0445 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:58.311956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3cd22308-ecb8-4fbc-a778-7387d41e0445 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:58.342873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3cd22308-ecb8-4fbc-a778-7387d41e0445 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050934 May 25 16:37:58.343472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 608/1233] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:58 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:37:59.360216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4fa1af15-20c3-4cad-be55-6615c4e7d699 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:37:59.361652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4fa1af15-20c3-4cad-be55-6615c4e7d699 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:37:59.372708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4fa1af15-20c3-4cad-be55-6615c4e7d699 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:59.373466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4fa1af15-20c3-4cad-be55-6615c4e7d699 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:59.374726 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4fa1af15-20c3-4cad-be55-6615c4e7d699 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:37:59.375515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4fa1af15-20c3-4cad-be55-6615c4e7d699 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:37:59.407798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4fa1af15-20c3-4cad-be55-6615c4e7d699 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054452 May 25 16:37:59.408572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 626/1234] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:37:59 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:00.424326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c13309bf-80dd-4262-be63-6d1135a2a522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:00.425899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c13309bf-80dd-4262-be63-6d1135a2a522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:00.435860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c13309bf-80dd-4262-be63-6d1135a2a522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:00.436569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c13309bf-80dd-4262-be63-6d1135a2a522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:00.437755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c13309bf-80dd-4262-be63-6d1135a2a522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:00.438427 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c13309bf-80dd-4262-be63-6d1135a2a522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:00.475753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c13309bf-80dd-4262-be63-6d1135a2a522 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058040 May 25 16:38:00.479304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 609/1235] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:00 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:01.491537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fe11dd0e-89af-4693-b41f-548ce035701a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:01.492970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fe11dd0e-89af-4693-b41f-548ce035701a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:01.504365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe11dd0e-89af-4693-b41f-548ce035701a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:01.505401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe11dd0e-89af-4693-b41f-548ce035701a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:01.506841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe11dd0e-89af-4693-b41f-548ce035701a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:01.507317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe11dd0e-89af-4693-b41f-548ce035701a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:01.537618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fe11dd0e-89af-4693-b41f-548ce035701a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052870 May 25 16:38:01.541796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 627/1236] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:01 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:02.554332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-85fef8fd-a292-42ae-a46e-a898ac7fedb0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:02.555286 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-85fef8fd-a292-42ae-a46e-a898ac7fedb0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:02.561640 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85fef8fd-a292-42ae-a46e-a898ac7fedb0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:02.562093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85fef8fd-a292-42ae-a46e-a898ac7fedb0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:02.562858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85fef8fd-a292-42ae-a46e-a898ac7fedb0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:02.563344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-85fef8fd-a292-42ae-a46e-a898ac7fedb0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:02.594146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-85fef8fd-a292-42ae-a46e-a898ac7fedb0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047110 May 25 16:38:02.594714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 610/1237] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:02 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:03.610375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1b1468fa-0fd0-4e4d-95a1-e7168c5f19e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:03.611897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1b1468fa-0fd0-4e4d-95a1-e7168c5f19e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:03.620652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b1468fa-0fd0-4e4d-95a1-e7168c5f19e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:03.621105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b1468fa-0fd0-4e4d-95a1-e7168c5f19e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:03.621876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b1468fa-0fd0-4e4d-95a1-e7168c5f19e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:03.622310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b1468fa-0fd0-4e4d-95a1-e7168c5f19e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:03.653377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1b1468fa-0fd0-4e4d-95a1-e7168c5f19e2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049686 May 25 16:38:03.654190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 628/1238] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:03 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:04.670695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-adf88acb-1cd6-41c4-b35e-e3b30376f9ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:04.674341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-adf88acb-1cd6-41c4-b35e-e3b30376f9ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:04.682358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adf88acb-1cd6-41c4-b35e-e3b30376f9ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:04.683404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adf88acb-1cd6-41c4-b35e-e3b30376f9ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:04.684455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adf88acb-1cd6-41c4-b35e-e3b30376f9ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:04.685182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-adf88acb-1cd6-41c4-b35e-e3b30376f9ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:04.718871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-adf88acb-1cd6-41c4-b35e-e3b30376f9ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055295 May 25 16:38:04.719611 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 611/1239] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:04 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:05.735771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d33568ad-970f-49e2-9f70-d7a1462bf1fc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:05.737478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d33568ad-970f-49e2-9f70-d7a1462bf1fc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:05.746170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d33568ad-970f-49e2-9f70-d7a1462bf1fc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:05.746629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d33568ad-970f-49e2-9f70-d7a1462bf1fc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:05.747551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d33568ad-970f-49e2-9f70-d7a1462bf1fc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:05.748038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d33568ad-970f-49e2-9f70-d7a1462bf1fc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:05.780052 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d33568ad-970f-49e2-9f70-d7a1462bf1fc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051373 May 25 16:38:05.780591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 629/1240] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:05 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:06.797012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f686e06d-6051-4f12-9368-0703b1b388e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:06.798701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f686e06d-6051-4f12-9368-0703b1b388e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:06.808788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f686e06d-6051-4f12-9368-0703b1b388e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:06.809530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f686e06d-6051-4f12-9368-0703b1b388e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:06.810784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f686e06d-6051-4f12-9368-0703b1b388e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:06.811555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f686e06d-6051-4f12-9368-0703b1b388e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:06.844572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f686e06d-6051-4f12-9368-0703b1b388e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054678 May 25 16:38:06.845347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 612/1241] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:06 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:07.860677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1be3b3fc-901c-4728-967e-272cc2c402e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:07.862165 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1be3b3fc-901c-4728-967e-272cc2c402e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:07.870512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1be3b3fc-901c-4728-967e-272cc2c402e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:07.870966 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1be3b3fc-901c-4728-967e-272cc2c402e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:07.871772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1be3b3fc-901c-4728-967e-272cc2c402e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:07.872201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1be3b3fc-901c-4728-967e-272cc2c402e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:07.902246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1be3b3fc-901c-4728-967e-272cc2c402e4 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048324 May 25 16:38:07.902822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 630/1242] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:07 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:08.919139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a8fe7f06-8ebf-4a8e-b54d-cd451a2d27d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:08.920623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a8fe7f06-8ebf-4a8e-b54d-cd451a2d27d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:08.932003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8fe7f06-8ebf-4a8e-b54d-cd451a2d27d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:08.932758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8fe7f06-8ebf-4a8e-b54d-cd451a2d27d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:08.934031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8fe7f06-8ebf-4a8e-b54d-cd451a2d27d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:08.934753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8fe7f06-8ebf-4a8e-b54d-cd451a2d27d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:08.968809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a8fe7f06-8ebf-4a8e-b54d-cd451a2d27d0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056746 May 25 16:38:08.969373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 613/1243] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:08 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:09.983970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b0c06fbd-4516-4016-93ff-b2a496a2c4ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:09.985371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b0c06fbd-4516-4016-93ff-b2a496a2c4ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:09.995810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0c06fbd-4516-4016-93ff-b2a496a2c4ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:09.996409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0c06fbd-4516-4016-93ff-b2a496a2c4ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:09.997447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0c06fbd-4516-4016-93ff-b2a496a2c4ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:09.997993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0c06fbd-4516-4016-93ff-b2a496a2c4ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:10.031448 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b0c06fbd-4516-4016-93ff-b2a496a2c4ed tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053520 May 25 16:38:10.031906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 631/1244] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:09 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:11.046705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a1a5fb3f-2364-4f86-a126-378d4d2e032f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:11.048249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a1a5fb3f-2364-4f86-a126-378d4d2e032f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:11.057392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1a5fb3f-2364-4f86-a126-378d4d2e032f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:11.057836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1a5fb3f-2364-4f86-a126-378d4d2e032f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:11.058629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1a5fb3f-2364-4f86-a126-378d4d2e032f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:11.059060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1a5fb3f-2364-4f86-a126-378d4d2e032f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:11.089953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a1a5fb3f-2364-4f86-a126-378d4d2e032f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050340 May 25 16:38:11.090513 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 614/1245] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:11 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:12.102520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-da740d95-cb61-40d3-99ef-f8d722ab99ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:12.103476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-da740d95-cb61-40d3-99ef-f8d722ab99ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:12.110423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da740d95-cb61-40d3-99ef-f8d722ab99ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:12.110876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da740d95-cb61-40d3-99ef-f8d722ab99ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:12.111680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da740d95-cb61-40d3-99ef-f8d722ab99ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:12.112117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da740d95-cb61-40d3-99ef-f8d722ab99ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:12.146924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-da740d95-cb61-40d3-99ef-f8d722ab99ad tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049242 May 25 16:38:12.147547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 632/1246] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:12 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:13.162605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-81ebd93a-7c38-47a6-a2cf-e315300ed927 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:13.164158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-81ebd93a-7c38-47a6-a2cf-e315300ed927 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:13.176159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-81ebd93a-7c38-47a6-a2cf-e315300ed927 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:13.177006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-81ebd93a-7c38-47a6-a2cf-e315300ed927 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:13.178363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-81ebd93a-7c38-47a6-a2cf-e315300ed927 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:13.178960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-81ebd93a-7c38-47a6-a2cf-e315300ed927 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:13.209601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-81ebd93a-7c38-47a6-a2cf-e315300ed927 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053972 May 25 16:38:13.210166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 615/1247] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:13 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:14.221484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d15d4810-a7b0-4767-8381-c0c8ce8ff0c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:14.222534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d15d4810-a7b0-4767-8381-c0c8ce8ff0c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:14.234523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d15d4810-a7b0-4767-8381-c0c8ce8ff0c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:14.234523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d15d4810-a7b0-4767-8381-c0c8ce8ff0c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:14.235271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d15d4810-a7b0-4767-8381-c0c8ce8ff0c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:14.235704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d15d4810-a7b0-4767-8381-c0c8ce8ff0c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:14.273279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d15d4810-a7b0-4767-8381-c0c8ce8ff0c3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056201 May 25 16:38:14.275475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 633/1248] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:14 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:15.288411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0eca545f-b331-418d-97b3-0dd26f9bd555 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:15.290269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0eca545f-b331-418d-97b3-0dd26f9bd555 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:15.301288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eca545f-b331-418d-97b3-0dd26f9bd555 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:15.302028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eca545f-b331-418d-97b3-0dd26f9bd555 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:15.303413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eca545f-b331-418d-97b3-0dd26f9bd555 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:15.303845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eca545f-b331-418d-97b3-0dd26f9bd555 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:15.333696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0eca545f-b331-418d-97b3-0dd26f9bd555 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052056 May 25 16:38:15.337848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 616/1249] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:15 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:16.349727 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fb146a0c-e236-43fe-b543-3d8883049dbb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:16.351120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fb146a0c-e236-43fe-b543-3d8883049dbb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:16.362225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb146a0c-e236-43fe-b543-3d8883049dbb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:16.362970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb146a0c-e236-43fe-b543-3d8883049dbb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:16.364283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb146a0c-e236-43fe-b543-3d8883049dbb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:16.365223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fb146a0c-e236-43fe-b543-3d8883049dbb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:16.396477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fb146a0c-e236-43fe-b543-3d8883049dbb tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053670 May 25 16:38:16.397327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 634/1250] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:16 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:17.412803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c62a2c84-38b9-48ab-9bfe-e78290328249 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:17.413876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c62a2c84-38b9-48ab-9bfe-e78290328249 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:17.422652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c62a2c84-38b9-48ab-9bfe-e78290328249 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:17.422798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c62a2c84-38b9-48ab-9bfe-e78290328249 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:17.423711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c62a2c84-38b9-48ab-9bfe-e78290328249 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:17.424213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c62a2c84-38b9-48ab-9bfe-e78290328249 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:17.457658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c62a2c84-38b9-48ab-9bfe-e78290328249 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051897 May 25 16:38:17.458226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 617/1251] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:17 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:18.473912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c0357c72-ba45-4642-9dc1-1935ff7983c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:18.475508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c0357c72-ba45-4642-9dc1-1935ff7983c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:18.486209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c0357c72-ba45-4642-9dc1-1935ff7983c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:18.486976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c0357c72-ba45-4642-9dc1-1935ff7983c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:18.488413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c0357c72-ba45-4642-9dc1-1935ff7983c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:18.489160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c0357c72-ba45-4642-9dc1-1935ff7983c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:18.522079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c0357c72-ba45-4642-9dc1-1935ff7983c6 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055000 May 25 16:38:18.522662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 635/1252] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:18 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:19.538023 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6fccb351-6655-4e89-8722-45cbcd8720ec tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:19.539108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6fccb351-6655-4e89-8722-45cbcd8720ec tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:19.542202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=30 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:19.548984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6fccb351-6655-4e89-8722-45cbcd8720ec tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:19.549521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6fccb351-6655-4e89-8722-45cbcd8720ec tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:19.550421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6fccb351-6655-4e89-8722-45cbcd8720ec tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:19.550935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6fccb351-6655-4e89-8722-45cbcd8720ec tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:19.554107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=90 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:19.585057 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6fccb351-6655-4e89-8722-45cbcd8720ec tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053740 May 25 16:38:19.585659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 618/1253] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:19 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:20.602084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-44fe5192-9c55-46b1-a51e-fb7b17209247 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:20.604683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-44fe5192-9c55-46b1-a51e-fb7b17209247 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:20.610219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=30 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:20.616148 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44fe5192-9c55-46b1-a51e-fb7b17209247 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:20.616610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44fe5192-9c55-46b1-a51e-fb7b17209247 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:20.617363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44fe5192-9c55-46b1-a51e-fb7b17209247 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:20.617795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44fe5192-9c55-46b1-a51e-fb7b17209247 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:20.620728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=90 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:20.648920 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-44fe5192-9c55-46b1-a51e-fb7b17209247 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053658 May 25 16:38:20.649493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 636/1254] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:20 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:21.664263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-df4e6bdc-26d9-4c89-a908-00daff815f81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:21.665723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-df4e6bdc-26d9-4c89-a908-00daff815f81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:21.674360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df4e6bdc-26d9-4c89-a908-00daff815f81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:21.674929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df4e6bdc-26d9-4c89-a908-00daff815f81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:21.675926 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df4e6bdc-26d9-4c89-a908-00daff815f81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:21.676456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df4e6bdc-26d9-4c89-a908-00daff815f81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:21.709677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-df4e6bdc-26d9-4c89-a908-00daff815f81 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051822 May 25 16:38:21.710252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 619/1255] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:21 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:38:22.721654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b2d7115d-16e4-4ebd-b10e-c57d17b8c94a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:22.722584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b2d7115d-16e4-4ebd-b10e-c57d17b8c94a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:22.729510 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2d7115d-16e4-4ebd-b10e-c57d17b8c94a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:22.730205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2d7115d-16e4-4ebd-b10e-c57d17b8c94a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:22.730710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2d7115d-16e4-4ebd-b10e-c57d17b8c94a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:22.731168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2d7115d-16e4-4ebd-b10e-c57d17b8c94a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:22.760901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b2d7115d-16e4-4ebd-b10e-c57d17b8c94a tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043265 May 25 16:38:22.761701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 637/1256] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:22 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:23.777100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9a244ae3-4f69-40f5-a5ed-c855623610b0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:23.778593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9a244ae3-4f69-40f5-a5ed-c855623610b0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:23.789998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9a244ae3-4f69-40f5-a5ed-c855623610b0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:23.790820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9a244ae3-4f69-40f5-a5ed-c855623610b0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:23.792155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9a244ae3-4f69-40f5-a5ed-c855623610b0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:23.792886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9a244ae3-4f69-40f5-a5ed-c855623610b0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:23.829326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9a244ae3-4f69-40f5-a5ed-c855623610b0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059019 May 25 16:38:23.830034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 620/1257] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:23 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:24.845188 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-20b91399-dc84-4047-b43e-4c66f7820a4f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:24.846740 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-20b91399-dc84-4047-b43e-4c66f7820a4f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:24.854985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20b91399-dc84-4047-b43e-4c66f7820a4f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:24.855488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20b91399-dc84-4047-b43e-4c66f7820a4f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:24.856226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20b91399-dc84-4047-b43e-4c66f7820a4f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:24.856657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20b91399-dc84-4047-b43e-4c66f7820a4f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:24.887197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-20b91399-dc84-4047-b43e-4c66f7820a4f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048607 May 25 16:38:24.888062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 638/1258] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:24 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:25.903379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e3daca6a-79a0-46d1-a288-6d4efef27836 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:25.904878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e3daca6a-79a0-46d1-a288-6d4efef27836 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:25.917159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3daca6a-79a0-46d1-a288-6d4efef27836 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:25.918164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3daca6a-79a0-46d1-a288-6d4efef27836 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:25.919506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3daca6a-79a0-46d1-a288-6d4efef27836 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:25.920302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3daca6a-79a0-46d1-a288-6d4efef27836 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:25.953899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e3daca6a-79a0-46d1-a288-6d4efef27836 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057313 May 25 16:38:25.954477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 621/1259] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:25 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:26.966474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5305cd43-f28d-4eb1-a615-b68fb45c1a9c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:26.967618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5305cd43-f28d-4eb1-a615-b68fb45c1a9c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:26.977855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5305cd43-f28d-4eb1-a615-b68fb45c1a9c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:26.978496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5305cd43-f28d-4eb1-a615-b68fb45c1a9c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:26.979292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5305cd43-f28d-4eb1-a615-b68fb45c1a9c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:26.979806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5305cd43-f28d-4eb1-a615-b68fb45c1a9c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:27.016096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5305cd43-f28d-4eb1-a615-b68fb45c1a9c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053619 May 25 16:38:27.017222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 639/1260] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:26 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:28.120505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-230dcf05-1e39-414c-841a-e452f400c813 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:28.121954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-230dcf05-1e39-414c-841a-e452f400c813 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:28.131447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-230dcf05-1e39-414c-841a-e452f400c813 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:28.132167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-230dcf05-1e39-414c-841a-e452f400c813 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:28.133384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-230dcf05-1e39-414c-841a-e452f400c813 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:28.134082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-230dcf05-1e39-414c-841a-e452f400c813 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:28.170984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-230dcf05-1e39-414c-841a-e452f400c813 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.146188 May 25 16:38:28.171637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 622/1261] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:28 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:29.182794 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e9da30d0-40c3-4034-8ed7-13bdf28cb45d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:29.183780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e9da30d0-40c3-4034-8ed7-13bdf28cb45d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:29.190384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e9da30d0-40c3-4034-8ed7-13bdf28cb45d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.190835 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e9da30d0-40c3-4034-8ed7-13bdf28cb45d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.192204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e9da30d0-40c3-4034-8ed7-13bdf28cb45d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.192643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e9da30d0-40c3-4034-8ed7-13bdf28cb45d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.222650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e9da30d0-40c3-4034-8ed7-13bdf28cb45d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043788 May 25 16:38:29.223458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 640/1262] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:29 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:29.397132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d48c593d-c10f-4ffa-b1e4-20847666e42e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:29.397662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d48c593d-c10f-4ffa-b1e4-20847666e42e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:29.404434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d48c593d-c10f-4ffa-b1e4-20847666e42e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.404889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d48c593d-c10f-4ffa-b1e4-20847666e42e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.405965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d48c593d-c10f-4ffa-b1e4-20847666e42e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.406409 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d48c593d-c10f-4ffa-b1e4-20847666e42e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.433183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d48c593d-c10f-4ffa-b1e4-20847666e42e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:38:29.521930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d48c593d-c10f-4ffa-b1e4-20847666e42e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "DELETE /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 204 len: 0 microversion: 2.1 time: 0.128316 May 25 16:38:29.523546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 623/1263] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:38:29 2022] DELETE /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:38:29.536429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1373a56e-ab37-4df6-9bf8-bad592bcaea1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:29.537020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1373a56e-ab37-4df6-9bf8-bad592bcaea1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:29.537162 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f18847bf-f026-40a1-aa27-1643ae1badf6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:29.537698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f18847bf-f026-40a1-aa27-1643ae1badf6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:29.543705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1373a56e-ab37-4df6-9bf8-bad592bcaea1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.544147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1373a56e-ab37-4df6-9bf8-bad592bcaea1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.544309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f18847bf-f026-40a1-aa27-1643ae1badf6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.544967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1373a56e-ab37-4df6-9bf8-bad592bcaea1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.545231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f18847bf-f026-40a1-aa27-1643ae1badf6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.545428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1373a56e-ab37-4df6-9bf8-bad592bcaea1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.546584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f18847bf-f026-40a1-aa27-1643ae1badf6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.547195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f18847bf-f026-40a1-aa27-1643ae1badf6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.572328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1373a56e-ab37-4df6-9bf8-bad592bcaea1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:38:29.657146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1373a56e-ab37-4df6-9bf8-bad592bcaea1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 204 len: 0 microversion: 2.1 time: 0.124484 May 25 16:38:29.661614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 624/1264] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:38:29 2022] DELETE /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:38:29.668606 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e223968d-e83d-4a02-8a0a-5b840ae27b7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:29.669077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e223968d-e83d-4a02-8a0a-5b840ae27b7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:29.675677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e223968d-e83d-4a02-8a0a-5b840ae27b7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.676176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e223968d-e83d-4a02-8a0a-5b840ae27b7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.677960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e223968d-e83d-4a02-8a0a-5b840ae27b7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.678449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e223968d-e83d-4a02-8a0a-5b840ae27b7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.704973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e223968d-e83d-4a02-8a0a-5b840ae27b7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:38:29.754112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f18847bf-f026-40a1-aa27-1643ae1badf6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:29.762216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f18847bf-f026-40a1-aa27-1643ae1badf6 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1673 microversion: 2.1 time: 0.230261 May 25 16:38:29.762803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 641/1265] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:29 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1673 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:29.777204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6c93c248-1b07-4fc4-b31c-f9709bb1458e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:29.777875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6c93c248-1b07-4fc4-b31c-f9709bb1458e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:29.779904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e223968d-e83d-4a02-8a0a-5b840ae27b7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 204 len: 0 microversion: 2.1 time: 0.116684 May 25 16:38:29.780584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 625/1266] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:38:29 2022] DELETE /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:38:29.787553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c93c248-1b07-4fc4-b31c-f9709bb1458e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.788007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c93c248-1b07-4fc4-b31c-f9709bb1458e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.788797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c93c248-1b07-4fc4-b31c-f9709bb1458e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.789233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c93c248-1b07-4fc4-b31c-f9709bb1458e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.790343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3e7ec5ab-2591-45b1-8e1e-98a1c06a6a0d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:29.791043 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3e7ec5ab-2591-45b1-8e1e-98a1c06a6a0d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:29.797878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e7ec5ab-2591-45b1-8e1e-98a1c06a6a0d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.798395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e7ec5ab-2591-45b1-8e1e-98a1c06a6a0d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.799207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e7ec5ab-2591-45b1-8e1e-98a1c06a6a0d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:29.799623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e7ec5ab-2591-45b1-8e1e-98a1c06a6a0d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:29.989703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6c93c248-1b07-4fc4-b31c-f9709bb1458e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:29.999195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6c93c248-1b07-4fc4-b31c-f9709bb1458e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.225924 May 25 16:38:29.999801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 642/1267] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:29 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:30.098912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3e7ec5ab-2591-45b1-8e1e-98a1c06a6a0d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:30.107026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3e7ec5ab-2591-45b1-8e1e-98a1c06a6a0d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.321709 May 25 16:38:30.107703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 626/1268] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:29 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:38:30.234046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5d138e5a-d421-4370-9bbd-4540fce0888e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:30.235039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5d138e5a-d421-4370-9bbd-4540fce0888e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:30.241607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d138e5a-d421-4370-9bbd-4540fce0888e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.242059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d138e5a-d421-4370-9bbd-4540fce0888e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.242824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d138e5a-d421-4370-9bbd-4540fce0888e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.243291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d138e5a-d421-4370-9bbd-4540fce0888e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.273927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5d138e5a-d421-4370-9bbd-4540fce0888e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043764 May 25 16:38:30.274487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 643/1269] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:30 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:30.338877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f1c4750c-d965-49da-b96c-067b37f62e0e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:30.339411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f1c4750c-d965-49da-b96c-067b37f62e0e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:30.346675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1c4750c-d965-49da-b96c-067b37f62e0e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.347228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1c4750c-d965-49da-b96c-067b37f62e0e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.348122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1c4750c-d965-49da-b96c-067b37f62e0e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.348634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f1c4750c-d965-49da-b96c-067b37f62e0e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.377473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f1c4750c-d965-49da-b96c-067b37f62e0e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:38:30.456092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f1c4750c-d965-49da-b96c-067b37f62e0e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "DELETE /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 204 len: 0 microversion: 2.20 time: 0.120225 May 25 16:38:30.457476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 627/1270] 10.222.0.45 () {60 vars in 1279 bytes} [Wed May 25 16:38:30 2022] DELETE /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 25 16:38:30.466213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:30.467058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:30.473410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.474154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.474946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.475461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.719320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:30.729259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:30.730838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:30.732286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-39c58152-6639-4274-83b9-f8218aa3d5f4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.270233 May 25 16:38:30.742979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 644/1271] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:30 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:30.764592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4bb3c1bf-ef87-47e5-b0d3-b91132fec82a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:30.766108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4bb3c1bf-ef87-47e5-b0d3-b91132fec82a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:30.771321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bb3c1bf-ef87-47e5-b0d3-b91132fec82a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.771741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bb3c1bf-ef87-47e5-b0d3-b91132fec82a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.772521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bb3c1bf-ef87-47e5-b0d3-b91132fec82a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.772948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bb3c1bf-ef87-47e5-b0d3-b91132fec82a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.775822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=4,nova_cell1:INSERT=4 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:30.780739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0e9c6caa-258e-48d6-b714-71c5d5813283 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:30.781603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0e9c6caa-258e-48d6-b714-71c5d5813283 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:30.788120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e9c6caa-258e-48d6-b714-71c5d5813283 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.789956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e9c6caa-258e-48d6-b714-71c5d5813283 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.789956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e9c6caa-258e-48d6-b714-71c5d5813283 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.790289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e9c6caa-258e-48d6-b714-71c5d5813283 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.802853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4bb3c1bf-ef87-47e5-b0d3-b91132fec82a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:38:30.866340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0e9c6caa-258e-48d6-b714-71c5d5813283 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4e4d39c5ed249bc9dc097ba4c24b3d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b36e59bfe72a49679b3b3657f5a65882', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:30.877847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0e9c6caa-258e-48d6-b714-71c5d5813283 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 200 len: 1605 microversion: 2.1 time: 0.101735 May 25 16:38:30.888159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4bb3c1bf-ef87-47e5-b0d3-b91132fec82a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 204 len: 0 microversion: 2.1 time: 0.127309 May 25 16:38:30.889444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 628/1272] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:38:30 2022] DELETE /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:38:30.895672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 645/1273] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:30 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 1605 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:30.898889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-821a0383-4990-4c71-b36b-82a3ca801e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:30.899740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-821a0383-4990-4c71-b36b-82a3ca801e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:30.906774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-821a0383-4990-4c71-b36b-82a3ca801e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.907263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-821a0383-4990-4c71-b36b-82a3ca801e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:30.908029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-821a0383-4990-4c71-b36b-82a3ca801e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:30.908638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-821a0383-4990-4c71-b36b-82a3ca801e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.122322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-821a0383-4990-4c71-b36b-82a3ca801e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:31.134667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-24a114f7-7a74-45cd-a563-802ce58f15f6 req-ee8f0016-91ff-4459-8c4a-bdd6a4f5b799 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c98878d3-b9b6-4688-abea-b14c2150fb4a", "name": "network-vif-deleted", "tag": "9beaea24-9678-4ef5-8aee-b08193e580ba"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:38:31.144352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-821a0383-4990-4c71-b36b-82a3ca801e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1728 microversion: 2.1 time: 0.250370 May 25 16:38:31.146816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 629/1274] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:30 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1728 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:31.167570 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-24a114f7-7a74-45cd-a563-802ce58f15f6 req-ee8f0016-91ff-4459-8c4a-bdd6a4f5b799 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.168289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-24a114f7-7a74-45cd-a563-802ce58f15f6 req-ee8f0016-91ff-4459-8c4a-bdd6a4f5b799 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.172414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-43dca9df-293e-4154-9392-f9b433b7cee3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:31.173345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-43dca9df-293e-4154-9392-f9b433b7cee3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:31.187531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43dca9df-293e-4154-9392-f9b433b7cee3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.188491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43dca9df-293e-4154-9392-f9b433b7cee3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.191413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43dca9df-293e-4154-9392-f9b433b7cee3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.192081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43dca9df-293e-4154-9392-f9b433b7cee3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.203486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-24a114f7-7a74-45cd-a563-802ce58f15f6 req-ee8f0016-91ff-4459-8c4a-bdd6a4f5b799 service nova] Creating event network-vif-deleted:9beaea24-9678-4ef5-8aee-b08193e580ba for instance c98878d3-b9b6-4688-abea-b14c2150fb4a on hv-cinder-84327 May 25 16:38:31.215521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-24a114f7-7a74-45cd-a563-802ce58f15f6 req-ee8f0016-91ff-4459-8c4a-bdd6a4f5b799 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.202765 May 25 16:38:31.216057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 646/1275] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:38:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:31.221307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ebe31efd-3819-4c05-ac50-68d58b913f8f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:31.222471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ebe31efd-3819-4c05-ac50-68d58b913f8f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:31.230532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebe31efd-3819-4c05-ac50-68d58b913f8f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.230532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebe31efd-3819-4c05-ac50-68d58b913f8f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.231335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebe31efd-3819-4c05-ac50-68d58b913f8f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.231759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebe31efd-3819-4c05-ac50-68d58b913f8f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.442056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-43dca9df-293e-4154-9392-f9b433b7cee3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:31.450351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-43dca9df-293e-4154-9392-f9b433b7cee3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.300581 May 25 16:38:31.451000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 630/1276] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:31 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:31.455781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1f137aa1-7a77-4b7d-ae31-242527e752c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:31.457042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1f137aa1-7a77-4b7d-ae31-242527e752c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:31.463585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ebe31efd-3819-4c05-ac50-68d58b913f8f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:31.472440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1f137aa1-7a77-4b7d-ae31-242527e752c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.472837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ebe31efd-3819-4c05-ac50-68d58b913f8f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.255630 May 25 16:38:31.473009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1f137aa1-7a77-4b7d-ae31-242527e752c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.473416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 647/1277] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:31 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:31.473673 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1f137aa1-7a77-4b7d-ae31-242527e752c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.474112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1f137aa1-7a77-4b7d-ae31-242527e752c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.477895 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e56dd000-4b81-4ee8-a3f0-45d63028c017 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] Calling method '.version_select at 0x7f4a3526df70>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:31.510266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e56dd000-4b81-4ee8-a3f0-45d63028c017 tempest-TestVolumeBootPattern-1840251582 tempest-TestVolumeBootPattern-1840251582-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-2139248485" status: 202 len: 0 microversion: 2.1 time: 0.035297 May 25 16:38:31.510854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 648/1278] 10.222.0.45 () {58 vars in 1262 bytes} [Wed May 25 16:38:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-2139248485 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:38:31.527430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1f137aa1-7a77-4b7d-ae31-242527e752c2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075293 May 25 16:38:31.528438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 631/1279] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:31 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:31.750421 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:31.751506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:31.763080 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.765108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.767114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.767747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.898417 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:31.899049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:31.905333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.907604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.909344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:31.910141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:31.932113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] HTTP exception thrown: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Request method failure captured: May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a HTTP/1.1 May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaSt3NdfdTff4OH_doadoVd4ihuLN8fcTpnktVD_XWHHevyVxY4tmWr46X1fVz2TuvDT5fSQ1evXFm6-DfkMDQmPYTRLtjUiNx8nVTeCIRoDB7T5LAC8qRTpjGqBOay7ujWQTOdtPe6UQNGpwsHetNPMu5dXLWOI1Kk0Q9IbllFii8 May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:38:31.934781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeTestJSON-755332015 May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeTestJSON-755332015-project May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: e4e4d39c5ed249bc9dc097ba4c24b3d2 May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeTestJSON-755332015-project May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:38:31.936472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'c98878d3-b9b6-4688-abea-b14c2150fb4a'} May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:38:31.937506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:38:31.939285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:38:31.941626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Returning 404 to user: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:38:31.942662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6cf2940c-aa56-4838-9837-dfb27d42eb82 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 404 len: 111 microversion: 2.1 time: 0.048663 May 25 16:38:31.943815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 632/1280] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:31 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 25 16:38:31.952850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a1418bf1-09ca-42a1-8fa9-29e01ea9b3e4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '.version_select at 0x7f4a300fa550>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:31.978474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a1418bf1-09ca-42a1-8fa9-29e01ea9b3e4 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-883348019" status: 202 len: 0 microversion: 2.1 time: 0.029262 May 25 16:38:31.983098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 633/1281] 10.222.0.45 () {58 vars in 1217 bytes} [Wed May 25 16:38:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-883348019 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:38:32.050776 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:32.061275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:32.063507 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:32.072742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-422c0b30-56ad-4b42-aae5-b84a9b0fb73e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.326571 May 25 16:38:32.073423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 649/1282] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:31 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:32.163109 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b6f29b35-cd0d-4bb1-a590-85cb4e9994d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:32.163740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b6f29b35-cd0d-4bb1-a590-85cb4e9994d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:32.172947 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b6f29b35-cd0d-4bb1-a590-85cb4e9994d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:32.173359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b6f29b35-cd0d-4bb1-a590-85cb4e9994d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:32.174136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b6f29b35-cd0d-4bb1-a590-85cb4e9994d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:32.174571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b6f29b35-cd0d-4bb1-a590-85cb4e9994d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:32.469009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-20399f19-09c9-43d0-9778-c4325aea2745 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:32.470356 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-20399f19-09c9-43d0-9778-c4325aea2745 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:32.477370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20399f19-09c9-43d0-9778-c4325aea2745 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:32.478131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20399f19-09c9-43d0-9778-c4325aea2745 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:32.479138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20399f19-09c9-43d0-9778-c4325aea2745 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:32.479873 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20399f19-09c9-43d0-9778-c4325aea2745 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:32.561605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b6f29b35-cd0d-4bb1-a590-85cb4e9994d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:32.572510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b6f29b35-cd0d-4bb1-a590-85cb4e9994d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1728 microversion: 2.1 time: 0.413019 May 25 16:38:32.573646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 634/1283] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:32 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1728 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:32.579255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1fe9b47e-ade6-488a-a256-4b9d834eefa0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:32.580113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1fe9b47e-ade6-488a-a256-4b9d834eefa0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:32.588994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fe9b47e-ade6-488a-a256-4b9d834eefa0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:32.589785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fe9b47e-ade6-488a-a256-4b9d834eefa0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:32.590602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fe9b47e-ade6-488a-a256-4b9d834eefa0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:32.591098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1fe9b47e-ade6-488a-a256-4b9d834eefa0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:33.147374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-20399f19-09c9-43d0-9778-c4325aea2745 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:33.156524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-20399f19-09c9-43d0-9778-c4325aea2745 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.691355 May 25 16:38:33.157397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 650/1284] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:32 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 693 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:33.164863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3beba67c-5c5b-466b-b4aa-2e2ecc22a328 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:33.169886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3beba67c-5c5b-466b-b4aa-2e2ecc22a328 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:33.179584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3beba67c-5c5b-466b-b4aa-2e2ecc22a328 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:33.180029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3beba67c-5c5b-466b-b4aa-2e2ecc22a328 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:33.180807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3beba67c-5c5b-466b-b4aa-2e2ecc22a328 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:33.181238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3beba67c-5c5b-466b-b4aa-2e2ecc22a328 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:33.221399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3beba67c-5c5b-466b-b4aa-2e2ecc22a328 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062726 May 25 16:38:33.225412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 651/1285] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:33 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:33.227307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:33.227933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:33.235977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:33.236421 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:33.237194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:33.237627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:33.257954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1fe9b47e-ade6-488a-a256-4b9d834eefa0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:33.266561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1fe9b47e-ade6-488a-a256-4b9d834eefa0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.690927 May 25 16:38:33.267316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 635/1286] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:32 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:33.537287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:33.549199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:33.551404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:33.553207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9fa9fbec-22b1-4350-ab21-081d4dca2062 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.329839 May 25 16:38:33.554003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 652/1287] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:33 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:33.592482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e5382dbe-0c9c-4d2b-9b98-d433718850a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:33.593993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e5382dbe-0c9c-4d2b-9b98-d433718850a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:33.602448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5382dbe-0c9c-4d2b-9b98-d433718850a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:33.603510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5382dbe-0c9c-4d2b-9b98-d433718850a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:33.604661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5382dbe-0c9c-4d2b-9b98-d433718850a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:33.605389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5382dbe-0c9c-4d2b-9b98-d433718850a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:33.733287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e5382dbe-0c9c-4d2b-9b98-d433718850a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:33.745833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e5382dbe-0c9c-4d2b-9b98-d433718850a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1660 microversion: 2.1 time: 0.158159 May 25 16:38:33.746679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 636/1288] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:33 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1660 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:33.815945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-c18cefc8-7409-4fe8-9ab8-77a11f6d28c0 req-20d3f5f6-f86b-424c-ab2e-26c77713bbfb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48341208-161c-454c-b88e-56a30f695cb7", "name": "network-vif-deleted", "tag": "e2ff0c6b-df5c-4cfe-b8f2-13db43352e9b"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:38:33.844905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-c18cefc8-7409-4fe8-9ab8-77a11f6d28c0 req-20d3f5f6-f86b-424c-ab2e-26c77713bbfb service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:33.847074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-c18cefc8-7409-4fe8-9ab8-77a11f6d28c0 req-20d3f5f6-f86b-424c-ab2e-26c77713bbfb service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:33.871496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-c18cefc8-7409-4fe8-9ab8-77a11f6d28c0 req-20d3f5f6-f86b-424c-ab2e-26c77713bbfb service nova] Creating event network-vif-deleted:e2ff0c6b-df5c-4cfe-b8f2-13db43352e9b for instance 48341208-161c-454c-b88e-56a30f695cb7 on hv-cinder-84327 May 25 16:38:33.882275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-c18cefc8-7409-4fe8-9ab8-77a11f6d28c0 req-20d3f5f6-f86b-424c-ab2e-26c77713bbfb service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069604 May 25 16:38:33.883452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 653/1289] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:38:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:34.177087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-25423942-48a3-46f2-a6fd-93e9f43e018a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:34.178219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-25423942-48a3-46f2-a6fd-93e9f43e018a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:34.188061 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25423942-48a3-46f2-a6fd-93e9f43e018a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.188879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25423942-48a3-46f2-a6fd-93e9f43e018a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.189959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25423942-48a3-46f2-a6fd-93e9f43e018a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.190630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-25423942-48a3-46f2-a6fd-93e9f43e018a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.233564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7ca7ad42-29dc-4f8e-8881-c6bf6e0e242b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:34.234832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7ca7ad42-29dc-4f8e-8881-c6bf6e0e242b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:34.246292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7ca7ad42-29dc-4f8e-8881-c6bf6e0e242b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.248168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7ca7ad42-29dc-4f8e-8881-c6bf6e0e242b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.249109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7ca7ad42-29dc-4f8e-8881-c6bf6e0e242b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.250581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7ca7ad42-29dc-4f8e-8881-c6bf6e0e242b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.301618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7ca7ad42-29dc-4f8e-8881-c6bf6e0e242b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073057 May 25 16:38:34.302565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 654/1290] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:34 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:38:34.307803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8edaac2f-4357-411c-9a66-d3b6c6be94f5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:34.308417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8edaac2f-4357-411c-9a66-d3b6c6be94f5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:34.315720 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8edaac2f-4357-411c-9a66-d3b6c6be94f5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.316384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8edaac2f-4357-411c-9a66-d3b6c6be94f5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.317196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8edaac2f-4357-411c-9a66-d3b6c6be94f5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.317656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8edaac2f-4357-411c-9a66-d3b6c6be94f5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.322726 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:34.517847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-25423942-48a3-46f2-a6fd-93e9f43e018a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:34.532578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-25423942-48a3-46f2-a6fd-93e9f43e018a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.362027 May 25 16:38:34.541604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 637/1291] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:34 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:34.573440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:34.575039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:34.585235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.585899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.586826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.587633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.601655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8edaac2f-4357-411c-9a66-d3b6c6be94f5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:34.622471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8edaac2f-4357-411c-9a66-d3b6c6be94f5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.318359 May 25 16:38:34.624369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 655/1292] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:34 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:34.765655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6f372091-cce6-4b5a-a5c0-6d68837d1789 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:34.766255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6f372091-cce6-4b5a-a5c0-6d68837d1789 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:34.773138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f372091-cce6-4b5a-a5c0-6d68837d1789 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.773589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f372091-cce6-4b5a-a5c0-6d68837d1789 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.774368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f372091-cce6-4b5a-a5c0-6d68837d1789 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:34.774812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6f372091-cce6-4b5a-a5c0-6d68837d1789 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:34.860455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6f372091-cce6-4b5a-a5c0-6d68837d1789 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:34.865074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:34.871694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6f372091-cce6-4b5a-a5c0-6d68837d1789 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.111437 May 25 16:38:34.874576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 656/1293] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:34 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:38:34.892090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:34.894036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:34.895279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0408a71b-d793-4855-acaf-593780d6bbfc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.327153 May 25 16:38:34.895844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 638/1294] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:34 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:38:35.313500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1ae3ec2b-1632-4f49-a4b9-bb36eaa4c623 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:35.314637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1ae3ec2b-1632-4f49-a4b9-bb36eaa4c623 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:35.321607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ae3ec2b-1632-4f49-a4b9-bb36eaa4c623 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.322743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ae3ec2b-1632-4f49-a4b9-bb36eaa4c623 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:35.322875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ae3ec2b-1632-4f49-a4b9-bb36eaa4c623 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.323360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ae3ec2b-1632-4f49-a4b9-bb36eaa4c623 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:35.359495 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1ae3ec2b-1632-4f49-a4b9-bb36eaa4c623 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049950 May 25 16:38:35.360547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 657/1295] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:35 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:35.550362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-75cc9351-17bb-48d1-9c79-e543e02fe22a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:35.551104 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-75cc9351-17bb-48d1-9c79-e543e02fe22a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:35.566880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-75cc9351-17bb-48d1-9c79-e543e02fe22a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.567356 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-75cc9351-17bb-48d1-9c79-e543e02fe22a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:35.568132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-75cc9351-17bb-48d1-9c79-e543e02fe22a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.568578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-75cc9351-17bb-48d1-9c79-e543e02fe22a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:35.643739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0dde4b55-d299-4328-9c62-93a48ff0370b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:35.645231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0dde4b55-d299-4328-9c62-93a48ff0370b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:35.654119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0dde4b55-d299-4328-9c62-93a48ff0370b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.655462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0dde4b55-d299-4328-9c62-93a48ff0370b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:35.657319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0dde4b55-d299-4328-9c62-93a48ff0370b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.657977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0dde4b55-d299-4328-9c62-93a48ff0370b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:35.822031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-75cc9351-17bb-48d1-9c79-e543e02fe22a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:35.840053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-75cc9351-17bb-48d1-9c79-e543e02fe22a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.293275 May 25 16:38:35.841328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 639/1296] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:35 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:35.868443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0dde4b55-d299-4328-9c62-93a48ff0370b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:35.895472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0dde4b55-d299-4328-9c62-93a48ff0370b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.256963 May 25 16:38:35.896059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 658/1297] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:35 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:38:35.912588 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8207b92c-8bff-4a40-8ce5-45f526c28d1f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:35.914211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8207b92c-8bff-4a40-8ce5-45f526c28d1f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:35.915227 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:35.915838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:35.921392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8207b92c-8bff-4a40-8ce5-45f526c28d1f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.922082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8207b92c-8bff-4a40-8ce5-45f526c28d1f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:35.923097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8207b92c-8bff-4a40-8ce5-45f526c28d1f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.923697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8207b92c-8bff-4a40-8ce5-45f526c28d1f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:35.924270 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.924733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:35.925623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:35.926058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:36.039892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8207b92c-8bff-4a40-8ce5-45f526c28d1f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:36.047974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8207b92c-8bff-4a40-8ce5-45f526c28d1f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.151654 May 25 16:38:36.048545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 640/1298] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:35 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:36.211469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:36.220869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:36.222709 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:36.223944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7a15ad33-6093-4c8e-a785-cc07f0fa98aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.313123 May 25 16:38:36.230768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 659/1299] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:35 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:36.372679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8cd70aca-13fd-4a8f-8feb-6871d1519ea5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:36.374468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8cd70aca-13fd-4a8f-8feb-6871d1519ea5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:36.381818 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8cd70aca-13fd-4a8f-8feb-6871d1519ea5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:36.382573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8cd70aca-13fd-4a8f-8feb-6871d1519ea5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:36.383384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8cd70aca-13fd-4a8f-8feb-6871d1519ea5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:36.384059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8cd70aca-13fd-4a8f-8feb-6871d1519ea5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:36.417650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8cd70aca-13fd-4a8f-8feb-6871d1519ea5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049969 May 25 16:38:36.421480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 641/1300] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:36 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:36.857246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6e03e7af-4922-496d-9c56-61dbf9160b86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:36.857900 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6e03e7af-4922-496d-9c56-61dbf9160b86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:36.864496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e03e7af-4922-496d-9c56-61dbf9160b86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:36.864978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e03e7af-4922-496d-9c56-61dbf9160b86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:36.865745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e03e7af-4922-496d-9c56-61dbf9160b86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:36.866176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6e03e7af-4922-496d-9c56-61dbf9160b86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:36.911947 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fd8da8c9-d462-48aa-8999-006b508548d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:36.913105 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fd8da8c9-d462-48aa-8999-006b508548d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:36.920604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd8da8c9-d462-48aa-8999-006b508548d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:36.921056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd8da8c9-d462-48aa-8999-006b508548d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:36.921822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd8da8c9-d462-48aa-8999-006b508548d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:36.922265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd8da8c9-d462-48aa-8999-006b508548d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:37.119875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6e03e7af-4922-496d-9c56-61dbf9160b86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:37.128139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6e03e7af-4922-496d-9c56-61dbf9160b86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.274944 May 25 16:38:37.128801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 660/1301] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:36 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:37.138235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a531b042-2cca-4bd4-8f5d-e4ac310e6b2c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:37.138235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a531b042-2cca-4bd4-8f5d-e4ac310e6b2c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:37.140114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a531b042-2cca-4bd4-8f5d-e4ac310e6b2c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:37.140771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a531b042-2cca-4bd4-8f5d-e4ac310e6b2c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:37.141589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a531b042-2cca-4bd4-8f5d-e4ac310e6b2c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:37.142084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a531b042-2cca-4bd4-8f5d-e4ac310e6b2c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:37.183307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fd8da8c9-d462-48aa-8999-006b508548d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:37.191891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fd8da8c9-d462-48aa-8999-006b508548d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.283685 May 25 16:38:37.192457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 642/1302] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:36 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:38:37.222576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a531b042-2cca-4bd4-8f5d-e4ac310e6b2c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:37.230829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a531b042-2cca-4bd4-8f5d-e4ac310e6b2c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.100990 May 25 16:38:37.231439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 661/1303] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:37 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:37.242123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:37.242836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:37.249436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:37.249884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:37.250715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:37.251147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:37.429618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b2ca65cf-6f38-4a37-8826-c22cf225226d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:37.430524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b2ca65cf-6f38-4a37-8826-c22cf225226d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:37.436676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2ca65cf-6f38-4a37-8826-c22cf225226d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:37.437130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2ca65cf-6f38-4a37-8826-c22cf225226d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:37.437909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2ca65cf-6f38-4a37-8826-c22cf225226d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:37.438347 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2ca65cf-6f38-4a37-8826-c22cf225226d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:37.467048 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:37.468481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b2ca65cf-6f38-4a37-8826-c22cf225226d tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042995 May 25 16:38:37.469070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 662/1304] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:37 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:37.476535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:37.478029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:37.479252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3192055f-7834-44ad-8c84-ca85174e101d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.241287 May 25 16:38:37.479801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 643/1305] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:37 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:38.145843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-81d7bc43-878b-42a7-bd84-6485f3a267ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:38.146454 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-81d7bc43-878b-42a7-bd84-6485f3a267ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:38.153252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-81d7bc43-878b-42a7-bd84-6485f3a267ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.153698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-81d7bc43-878b-42a7-bd84-6485f3a267ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.154459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-81d7bc43-878b-42a7-bd84-6485f3a267ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.154892 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-81d7bc43-878b-42a7-bd84-6485f3a267ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.210661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-28d91b27-a244-4dd4-ab4c-58ba7259ddb0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:38.211889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-28d91b27-a244-4dd4-ab4c-58ba7259ddb0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:38.220495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28d91b27-a244-4dd4-ab4c-58ba7259ddb0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.223809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28d91b27-a244-4dd4-ab4c-58ba7259ddb0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.228237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28d91b27-a244-4dd4-ab4c-58ba7259ddb0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.228698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28d91b27-a244-4dd4-ab4c-58ba7259ddb0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.434386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-81d7bc43-878b-42a7-bd84-6485f3a267ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:38.443590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-81d7bc43-878b-42a7-bd84-6485f3a267ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.301583 May 25 16:38:38.444324 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 663/1306] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:38 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:38.457698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9b67e182-cf3f-4219-9c74-4c947ba32910 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:38.458405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9b67e182-cf3f-4219-9c74-4c947ba32910 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:38.465620 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b67e182-cf3f-4219-9c74-4c947ba32910 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.466204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b67e182-cf3f-4219-9c74-4c947ba32910 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.467214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b67e182-cf3f-4219-9c74-4c947ba32910 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.467798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b67e182-cf3f-4219-9c74-4c947ba32910 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.501702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-28d91b27-a244-4dd4-ab4c-58ba7259ddb0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:38.504476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:38.511748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-28d91b27-a244-4dd4-ab4c-58ba7259ddb0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.305443 May 25 16:38:38.512414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 644/1307] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:38 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:38.517481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2c4d4dee-919b-4dcd-bbdf-f38261a9ade0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:38.518515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2c4d4dee-919b-4dcd-bbdf-f38261a9ade0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:38.526438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c4d4dee-919b-4dcd-bbdf-f38261a9ade0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.527086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c4d4dee-919b-4dcd-bbdf-f38261a9ade0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.528037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c4d4dee-919b-4dcd-bbdf-f38261a9ade0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.528624 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c4d4dee-919b-4dcd-bbdf-f38261a9ade0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.552640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9b67e182-cf3f-4219-9c74-4c947ba32910 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:38.561026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9b67e182-cf3f-4219-9c74-4c947ba32910 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.115616 May 25 16:38:38.561608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 664/1308] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:38 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:38.562665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2c4d4dee-919b-4dcd-bbdf-f38261a9ade0 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048388 May 25 16:38:38.566424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:38.566785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 645/1309] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:38 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:38.567025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:38.573328 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.573771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.574853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:38.575315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:38.807802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:38.816564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:38.818157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:38.819376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d8a42b34-7547-458c-b386-c32abb6b10d4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.256645 May 25 16:38:38.820006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 665/1310] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:38 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:39.461971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0e4100ec-454d-4f94-8d42-bb00355eb0d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:39.463332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0e4100ec-454d-4f94-8d42-bb00355eb0d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:39.470968 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e4100ec-454d-4f94-8d42-bb00355eb0d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:39.471722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e4100ec-454d-4f94-8d42-bb00355eb0d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:39.472723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e4100ec-454d-4f94-8d42-bb00355eb0d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:39.473151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e4100ec-454d-4f94-8d42-bb00355eb0d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:39.529272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-05fdcc91-f231-4497-9756-a2345e443ef6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:39.529989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-05fdcc91-f231-4497-9756-a2345e443ef6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:39.537242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-05fdcc91-f231-4497-9756-a2345e443ef6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:39.538030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-05fdcc91-f231-4497-9756-a2345e443ef6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:39.539261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-05fdcc91-f231-4497-9756-a2345e443ef6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:39.539837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-05fdcc91-f231-4497-9756-a2345e443ef6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:40.059291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0e4100ec-454d-4f94-8d42-bb00355eb0d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:40.062327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-05fdcc91-f231-4497-9756-a2345e443ef6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:40.068341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0e4100ec-454d-4f94-8d42-bb00355eb0d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.610856 May 25 16:38:40.068921 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 646/1311] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:39 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:40.083201 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ace01ec7-a3ee-4eb4-bae0-abe330a9e02e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:40.084404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ace01ec7-a3ee-4eb4-bae0-abe330a9e02e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:40.087669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-05fdcc91-f231-4497-9756-a2345e443ef6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.561975 May 25 16:38:40.088472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 666/1312] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:39 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:38:40.091049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ace01ec7-a3ee-4eb4-bae0-abe330a9e02e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:40.091537 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ace01ec7-a3ee-4eb4-bae0-abe330a9e02e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:40.092295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ace01ec7-a3ee-4eb4-bae0-abe330a9e02e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:40.092720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ace01ec7-a3ee-4eb4-bae0-abe330a9e02e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:40.093451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-077001db-2d8b-4f36-b86c-01151c71d657 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:40.094051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-077001db-2d8b-4f36-b86c-01151c71d657 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:40.100446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-077001db-2d8b-4f36-b86c-01151c71d657 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:40.101205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-077001db-2d8b-4f36-b86c-01151c71d657 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:40.101973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-077001db-2d8b-4f36-b86c-01151c71d657 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:40.102394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-077001db-2d8b-4f36-b86c-01151c71d657 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:40.126020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ace01ec7-a3ee-4eb4-bae0-abe330a9e02e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056030 May 25 16:38:40.126594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 647/1313] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:40 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:40.137098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:40.137237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:40.143645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:40.144099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:40.144867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:40.145305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:40.229689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-077001db-2d8b-4f36-b86c-01151c71d657 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:40.239951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-077001db-2d8b-4f36-b86c-01151c71d657 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.150086 May 25 16:38:40.240701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 667/1314] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:40 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:40.389531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:40.398972 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:40.400545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:40.401948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b0ffc666-0597-451d-b377-44a1d29718c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.274122 May 25 16:38:40.412831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 648/1315] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:40 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:41.086219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2868848e-a26b-481d-a3e0-6b4a00553049 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:41.086832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2868848e-a26b-481d-a3e0-6b4a00553049 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:41.093709 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2868848e-a26b-481d-a3e0-6b4a00553049 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.094178 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2868848e-a26b-481d-a3e0-6b4a00553049 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.094944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2868848e-a26b-481d-a3e0-6b4a00553049 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.095419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2868848e-a26b-481d-a3e0-6b4a00553049 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.105133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-611e24a5-2fa8-4d5d-96fd-c6e21491822b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:41.105726 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-611e24a5-2fa8-4d5d-96fd-c6e21491822b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:41.113340 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-611e24a5-2fa8-4d5d-96fd-c6e21491822b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.113811 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-611e24a5-2fa8-4d5d-96fd-c6e21491822b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.114547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-611e24a5-2fa8-4d5d-96fd-c6e21491822b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.115076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-611e24a5-2fa8-4d5d-96fd-c6e21491822b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.303546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2868848e-a26b-481d-a3e0-6b4a00553049 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:41.310462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-611e24a5-2fa8-4d5d-96fd-c6e21491822b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:41.318766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2868848e-a26b-481d-a3e0-6b4a00553049 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.236603 May 25 16:38:41.318970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-611e24a5-2fa8-4d5d-96fd-c6e21491822b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217413 May 25 16:38:41.319372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 668/1316] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:41 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:41.320286 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 649/1317] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:41 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:38:41.324904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c1d57ba1-1751-4c1a-aba2-c9c97bdd3bc5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:41.325901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c1d57ba1-1751-4c1a-aba2-c9c97bdd3bc5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:41.332095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1d57ba1-1751-4c1a-aba2-c9c97bdd3bc5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.332538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1d57ba1-1751-4c1a-aba2-c9c97bdd3bc5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.333083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f96c380a-6db1-47d4-8cef-f765970c9f79 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:41.333297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1d57ba1-1751-4c1a-aba2-c9c97bdd3bc5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.333988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f96c380a-6db1-47d4-8cef-f765970c9f79 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:41.335111 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1d57ba1-1751-4c1a-aba2-c9c97bdd3bc5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.340634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f96c380a-6db1-47d4-8cef-f765970c9f79 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.341300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f96c380a-6db1-47d4-8cef-f765970c9f79 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.342121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f96c380a-6db1-47d4-8cef-f765970c9f79 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.342770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f96c380a-6db1-47d4-8cef-f765970c9f79 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.365339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c1d57ba1-1751-4c1a-aba2-c9c97bdd3bc5 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044883 May 25 16:38:41.365912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 669/1318] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:41 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:41.404382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f96c380a-6db1-47d4-8cef-f765970c9f79 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:41.412602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f96c380a-6db1-47d4-8cef-f765970c9f79 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091196 May 25 16:38:41.413161 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 650/1319] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:41 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:41.422061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:41.422671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:41.428821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.429260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.430248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:41.430744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:41.573854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:41.582943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:41.584787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:41.585962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dc598986-8e93-432d-830a-02c6a2d20745 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.169460 May 25 16:38:41.586548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 670/1320] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:42.336284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-49d75aa7-781f-4fb9-be25-7aa82df93255 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:42.336989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ff15da3e-53b4-441e-95db-16720887429e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:42.337383 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-49d75aa7-781f-4fb9-be25-7aa82df93255 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:42.337508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ff15da3e-53b4-441e-95db-16720887429e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:42.344081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49d75aa7-781f-4fb9-be25-7aa82df93255 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.344268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff15da3e-53b4-441e-95db-16720887429e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.344527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49d75aa7-781f-4fb9-be25-7aa82df93255 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.344705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff15da3e-53b4-441e-95db-16720887429e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.345266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49d75aa7-781f-4fb9-be25-7aa82df93255 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.345454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff15da3e-53b4-441e-95db-16720887429e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.345693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49d75aa7-781f-4fb9-be25-7aa82df93255 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.345895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff15da3e-53b4-441e-95db-16720887429e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.538918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-49d75aa7-781f-4fb9-be25-7aa82df93255 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:42.541522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:42.546359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ff15da3e-53b4-441e-95db-16720887429e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:42.549052 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-49d75aa7-781f-4fb9-be25-7aa82df93255 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.216903 May 25 16:38:42.549611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 671/1321] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:42 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:42.554656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-be4dd2f9-3097-4245-8714-eb11c4052bb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:42.555213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ff15da3e-53b4-441e-95db-16720887429e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.223074 May 25 16:38:42.555862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-be4dd2f9-3097-4245-8714-eb11c4052bb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:42.559100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 651/1322] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:42 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:42.562174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be4dd2f9-3097-4245-8714-eb11c4052bb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.562622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be4dd2f9-3097-4245-8714-eb11c4052bb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.563403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be4dd2f9-3097-4245-8714-eb11c4052bb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.563757 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9ec8b506-52dc-4073-8dfc-4519f09c7f3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:42.563889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be4dd2f9-3097-4245-8714-eb11c4052bb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.564365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9ec8b506-52dc-4073-8dfc-4519f09c7f3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:42.566140 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=5,nova_api:DELETE=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:42.571665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ec8b506-52dc-4073-8dfc-4519f09c7f3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.572145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ec8b506-52dc-4073-8dfc-4519f09c7f3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.572919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ec8b506-52dc-4073-8dfc-4519f09c7f3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.573395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ec8b506-52dc-4073-8dfc-4519f09c7f3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.594640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-be4dd2f9-3097-4245-8714-eb11c4052bb7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043694 May 25 16:38:42.595370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 672/1323] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:42 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:42.604112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:42.604707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:42.611145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.611628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.612391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:42.612822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:42.637217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9ec8b506-52dc-4073-8dfc-4519f09c7f3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:42.645859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9ec8b506-52dc-4073-8dfc-4519f09c7f3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085715 May 25 16:38:42.646462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 652/1324] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:42 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:42.741296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:42.750046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:42.751581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:42.752756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-042966de-3c0c-4c44-8e36-af9f610d554b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.152395 May 25 16:38:42.753313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 673/1325] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:43.570130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8c1cd41c-5302-455f-a968-f21c9f3da932 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:43.570868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8c1cd41c-5302-455f-a968-f21c9f3da932 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:43.575440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-12da04bf-92d6-4807-8303-cc01890899b9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:43.576224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-12da04bf-92d6-4807-8303-cc01890899b9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:43.579395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c1cd41c-5302-455f-a968-f21c9f3da932 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.579959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c1cd41c-5302-455f-a968-f21c9f3da932 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.580910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c1cd41c-5302-455f-a968-f21c9f3da932 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.581453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c1cd41c-5302-455f-a968-f21c9f3da932 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.584093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12da04bf-92d6-4807-8303-cc01890899b9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.584693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12da04bf-92d6-4807-8303-cc01890899b9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.585661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12da04bf-92d6-4807-8303-cc01890899b9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.586300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12da04bf-92d6-4807-8303-cc01890899b9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.780804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8c1cd41c-5302-455f-a968-f21c9f3da932 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:43.783753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-12da04bf-92d6-4807-8303-cc01890899b9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:43.789095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8c1cd41c-5302-455f-a968-f21c9f3da932 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.224956 May 25 16:38:43.789974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 653/1326] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:43 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:43.792255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-12da04bf-92d6-4807-8303-cc01890899b9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.222378 May 25 16:38:43.794457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 674/1327] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:43 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:43.794622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b845bbc3-f7af-497d-be37-27f2af092354 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:43.795544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b845bbc3-f7af-497d-be37-27f2af092354 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:43.798562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a94e1695-1a51-47fd-8c1e-60e0246f894b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:43.799383 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a94e1695-1a51-47fd-8c1e-60e0246f894b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:43.801446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b845bbc3-f7af-497d-be37-27f2af092354 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.801889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b845bbc3-f7af-497d-be37-27f2af092354 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.802882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b845bbc3-f7af-497d-be37-27f2af092354 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.803495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b845bbc3-f7af-497d-be37-27f2af092354 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.805796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a94e1695-1a51-47fd-8c1e-60e0246f894b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.806241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a94e1695-1a51-47fd-8c1e-60e0246f894b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.807034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a94e1695-1a51-47fd-8c1e-60e0246f894b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.807504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a94e1695-1a51-47fd-8c1e-60e0246f894b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.833293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b845bbc3-f7af-497d-be37-27f2af092354 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042298 May 25 16:38:43.834144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 654/1328] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:43 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:43.839252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:43.839847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:43.846030 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.846479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.847252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:43.847692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:43.870721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a94e1695-1a51-47fd-8c1e-60e0246f894b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:43.879066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a94e1695-1a51-47fd-8c1e-60e0246f894b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084874 May 25 16:38:43.879660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 675/1329] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:43 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:43.982427 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:43.991385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:43.992883 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:43.994078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-49c87065-9f54-4155-b45d-91475c6cfda3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.158367 May 25 16:38:43.994636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 655/1330] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:44.806934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2b9b8c80-246a-4c3f-8d77-859f4e73ade2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:44.807562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2b9b8c80-246a-4c3f-8d77-859f4e73ade2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:44.809628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0839764b-60ab-4e76-b538-3facf5f39c4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:44.810223 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0839764b-60ab-4e76-b538-3facf5f39c4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:44.814233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b9b8c80-246a-4c3f-8d77-859f4e73ade2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:44.814669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b9b8c80-246a-4c3f-8d77-859f4e73ade2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:44.815481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b9b8c80-246a-4c3f-8d77-859f4e73ade2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:44.815912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b9b8c80-246a-4c3f-8d77-859f4e73ade2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:44.817237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0839764b-60ab-4e76-b538-3facf5f39c4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:44.817696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0839764b-60ab-4e76-b538-3facf5f39c4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:44.818461 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0839764b-60ab-4e76-b538-3facf5f39c4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:44.818894 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0839764b-60ab-4e76-b538-3facf5f39c4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:45.006876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2b9b8c80-246a-4c3f-8d77-859f4e73ade2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:45.010471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0839764b-60ab-4e76-b538-3facf5f39c4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:45.015587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2b9b8c80-246a-4c3f-8d77-859f4e73ade2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.213066 May 25 16:38:45.016238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 676/1331] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:44 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:45.019952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0839764b-60ab-4e76-b538-3facf5f39c4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.214055 May 25 16:38:45.021010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 656/1332] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:44 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:45.021182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-02362ce7-a7f7-4639-b16a-00c0441e8056 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:45.022189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-02362ce7-a7f7-4639-b16a-00c0441e8056 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:45.026273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5949ac89-4328-475d-a835-da71778c969e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:45.026879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5949ac89-4328-475d-a835-da71778c969e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:45.028411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02362ce7-a7f7-4639-b16a-00c0441e8056 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:45.028856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02362ce7-a7f7-4639-b16a-00c0441e8056 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:45.029868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02362ce7-a7f7-4639-b16a-00c0441e8056 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:45.030058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02362ce7-a7f7-4639-b16a-00c0441e8056 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:45.033228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5949ac89-4328-475d-a835-da71778c969e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:45.033723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5949ac89-4328-475d-a835-da71778c969e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:45.034520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5949ac89-4328-475d-a835-da71778c969e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:45.034962 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5949ac89-4328-475d-a835-da71778c969e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:45.060285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-02362ce7-a7f7-4639-b16a-00c0441e8056 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042881 May 25 16:38:45.060771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 677/1333] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:45 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:45.065294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:45.065901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:45.072372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:45.072833 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:45.073597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:45.074011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:45.098060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5949ac89-4328-475d-a835-da71778c969e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:45.106219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5949ac89-4328-475d-a835-da71778c969e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083927 May 25 16:38:45.106788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 657/1334] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:45 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:45.208217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:45.217025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:45.218736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:45.219923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d9d33edc-6d87-4fc9-8843-2ff2d7f790b5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.158239 May 25 16:38:45.220468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 678/1335] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:45 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:46.033467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7b60f068-7180-42bc-8713-d061fda089eb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:46.034105 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7b60f068-7180-42bc-8713-d061fda089eb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:46.038024 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e4dd22a1-83a4-4d24-ade2-b06a190cf29e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:46.038640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e4dd22a1-83a4-4d24-ade2-b06a190cf29e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:46.041492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b60f068-7180-42bc-8713-d061fda089eb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.042021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b60f068-7180-42bc-8713-d061fda089eb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.042794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b60f068-7180-42bc-8713-d061fda089eb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.043267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7b60f068-7180-42bc-8713-d061fda089eb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.045375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4dd22a1-83a4-4d24-ade2-b06a190cf29e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.045952 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4dd22a1-83a4-4d24-ade2-b06a190cf29e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.046721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4dd22a1-83a4-4d24-ade2-b06a190cf29e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.047174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e4dd22a1-83a4-4d24-ade2-b06a190cf29e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.249911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e4dd22a1-83a4-4d24-ade2-b06a190cf29e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:46.257166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7b60f068-7180-42bc-8713-d061fda089eb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:46.258810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e4dd22a1-83a4-4d24-ade2-b06a190cf29e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.225191 May 25 16:38:46.259408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 679/1336] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:46 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:46.264872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c7ac28c2-044a-40e9-acb2-6ce97f1b9181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:46.265857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c7ac28c2-044a-40e9-acb2-6ce97f1b9181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:46.265977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7b60f068-7180-42bc-8713-d061fda089eb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.236429 May 25 16:38:46.266667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 658/1337] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:46 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:46.272903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7ac28c2-044a-40e9-acb2-6ce97f1b9181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.273353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7ac28c2-044a-40e9-acb2-6ce97f1b9181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.274117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7ac28c2-044a-40e9-acb2-6ce97f1b9181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.274548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7ac28c2-044a-40e9-acb2-6ce97f1b9181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.274706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2e1b9d17-7a06-4e54-953a-3cd7c51a68f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:46.275396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2e1b9d17-7a06-4e54-953a-3cd7c51a68f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:46.281979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e1b9d17-7a06-4e54-953a-3cd7c51a68f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.282438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e1b9d17-7a06-4e54-953a-3cd7c51a68f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.283624 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e1b9d17-7a06-4e54-953a-3cd7c51a68f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.283997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e1b9d17-7a06-4e54-953a-3cd7c51a68f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.305660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c7ac28c2-044a-40e9-acb2-6ce97f1b9181 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045058 May 25 16:38:46.306238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 680/1338] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:46 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:46.310951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:46.311590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:46.317716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.318245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.319082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:46.319616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:46.350051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2e1b9d17-7a06-4e54-953a-3cd7c51a68f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:46.358944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2e1b9d17-7a06-4e54-953a-3cd7c51a68f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088462 May 25 16:38:46.359476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 659/1339] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:46 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:46.453341 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:46.461882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:46.463483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:46.464645 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-699ba59a-dc83-4182-a592-b39a8cbdc881 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.157400 May 25 16:38:46.465202 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 681/1340] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:46 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:47.281599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-de16024d-363a-469e-80cb-cae948ade519 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:47.282708 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-de16024d-363a-469e-80cb-cae948ade519 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:47.288033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2137dbcc-0684-4299-9e7d-85d7abfd522c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:47.288946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2137dbcc-0684-4299-9e7d-85d7abfd522c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:47.291933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de16024d-363a-469e-80cb-cae948ade519 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.292450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de16024d-363a-469e-80cb-cae948ade519 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.293305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de16024d-363a-469e-80cb-cae948ade519 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.293771 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de16024d-363a-469e-80cb-cae948ade519 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.296050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2137dbcc-0684-4299-9e7d-85d7abfd522c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.296492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2137dbcc-0684-4299-9e7d-85d7abfd522c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.297243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2137dbcc-0684-4299-9e7d-85d7abfd522c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.297673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2137dbcc-0684-4299-9e7d-85d7abfd522c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.487647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2137dbcc-0684-4299-9e7d-85d7abfd522c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:47.492095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-de16024d-363a-469e-80cb-cae948ade519 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:47.496031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2137dbcc-0684-4299-9e7d-85d7abfd522c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.214582 May 25 16:38:47.496609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 682/1341] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:47 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:47.500621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-de16024d-363a-469e-80cb-cae948ade519 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.226503 May 25 16:38:47.501360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-db8ed3e9-9d74-43c0-894c-a5652f23b069 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:47.501504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 660/1342] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:47 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:47.502122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-db8ed3e9-9d74-43c0-894c-a5652f23b069 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:47.509180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-381a9d35-ff78-462b-9340-cdb6aa498b4c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:47.509794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-381a9d35-ff78-462b-9340-cdb6aa498b4c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:47.512930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db8ed3e9-9d74-43c0-894c-a5652f23b069 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.513426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db8ed3e9-9d74-43c0-894c-a5652f23b069 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.514222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db8ed3e9-9d74-43c0-894c-a5652f23b069 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.514720 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db8ed3e9-9d74-43c0-894c-a5652f23b069 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.515815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-381a9d35-ff78-462b-9340-cdb6aa498b4c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.516262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-381a9d35-ff78-462b-9340-cdb6aa498b4c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.517017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-381a9d35-ff78-462b-9340-cdb6aa498b4c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.517449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-381a9d35-ff78-462b-9340-cdb6aa498b4c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.532405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:47.547573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-db8ed3e9-9d74-43c0-894c-a5652f23b069 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049856 May 25 16:38:47.548212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 683/1343] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:47 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:47.553284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:47.553953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:47.560432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.560919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.561774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:47.562232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:47.580253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-381a9d35-ff78-462b-9340-cdb6aa498b4c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:47.588418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-381a9d35-ff78-462b-9340-cdb6aa498b4c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.082861 May 25 16:38:47.588952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 661/1344] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:47 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:47.706805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:47.716065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:47.717610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:47.718818 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1c284351-e933-4ea8-9bed-fb47434bdf82 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.169643 May 25 16:38:47.719501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 684/1345] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:48.517532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-88e0173b-f8e9-463c-be8c-1afee9703fbe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:48.518544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-88e0173b-f8e9-463c-be8c-1afee9703fbe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:48.521826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e6c178ab-9cd5-464c-8467-726bcce6309c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:48.522704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e6c178ab-9cd5-464c-8467-726bcce6309c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:48.525280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:48.527024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88e0173b-f8e9-463c-be8c-1afee9703fbe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.527566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88e0173b-f8e9-463c-be8c-1afee9703fbe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.528340 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88e0173b-f8e9-463c-be8c-1afee9703fbe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.528779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88e0173b-f8e9-463c-be8c-1afee9703fbe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.531003 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6c178ab-9cd5-464c-8467-726bcce6309c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.531537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6c178ab-9cd5-464c-8467-726bcce6309c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.532383 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6c178ab-9cd5-464c-8467-726bcce6309c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.532845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6c178ab-9cd5-464c-8467-726bcce6309c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.723844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-88e0173b-f8e9-463c-be8c-1afee9703fbe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:48.728126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e6c178ab-9cd5-464c-8467-726bcce6309c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:48.732608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-88e0173b-f8e9-463c-be8c-1afee9703fbe tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221442 May 25 16:38:48.733190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 662/1346] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:48 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:48.736373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e6c178ab-9cd5-464c-8467-726bcce6309c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.221264 May 25 16:38:48.736938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 685/1347] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:48 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:48.739413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8c45314b-82c0-4b76-961d-dfb70448a486 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:48.740604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8c45314b-82c0-4b76-961d-dfb70448a486 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:48.742332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-87fb34bf-e51c-4e44-9d35-79f6c6508f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:48.743376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-87fb34bf-e51c-4e44-9d35-79f6c6508f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:48.747088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c45314b-82c0-4b76-961d-dfb70448a486 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.747589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c45314b-82c0-4b76-961d-dfb70448a486 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.748373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c45314b-82c0-4b76-961d-dfb70448a486 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.748810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8c45314b-82c0-4b76-961d-dfb70448a486 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.750058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87fb34bf-e51c-4e44-9d35-79f6c6508f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.750576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87fb34bf-e51c-4e44-9d35-79f6c6508f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.751439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87fb34bf-e51c-4e44-9d35-79f6c6508f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.751975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87fb34bf-e51c-4e44-9d35-79f6c6508f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.778416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8c45314b-82c0-4b76-961d-dfb70448a486 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044185 May 25 16:38:48.778988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 663/1348] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:48 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:48.783396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:48.784003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:48.790321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.790779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.791569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:48.792009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:48.814701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-87fb34bf-e51c-4e44-9d35-79f6c6508f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:48.823170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-87fb34bf-e51c-4e44-9d35-79f6c6508f89 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085027 May 25 16:38:48.823750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 686/1349] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:48 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:48.928829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:48.942015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:48.944029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:48.945193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-31343381-5bae-42de-9ace-71ec15e73252 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.165236 May 25 16:38:48.945760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 664/1350] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:49.754649 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ddddf217-f081-4ac2-a1b9-78baa95c3b36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:49.755718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ddddf217-f081-4ac2-a1b9-78baa95c3b36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:49.756746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2a13a308-8d7b-4310-beb8-3a8a2d12600d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:49.757383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2a13a308-8d7b-4310-beb8-3a8a2d12600d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:49.762141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ddddf217-f081-4ac2-a1b9-78baa95c3b36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:49.762593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ddddf217-f081-4ac2-a1b9-78baa95c3b36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:49.763389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ddddf217-f081-4ac2-a1b9-78baa95c3b36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:49.763817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ddddf217-f081-4ac2-a1b9-78baa95c3b36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:49.764741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a13a308-8d7b-4310-beb8-3a8a2d12600d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:49.765236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a13a308-8d7b-4310-beb8-3a8a2d12600d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:49.766101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a13a308-8d7b-4310-beb8-3a8a2d12600d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:49.766583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a13a308-8d7b-4310-beb8-3a8a2d12600d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:49.954283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ddddf217-f081-4ac2-a1b9-78baa95c3b36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:49.958363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2a13a308-8d7b-4310-beb8-3a8a2d12600d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:49.962747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ddddf217-f081-4ac2-a1b9-78baa95c3b36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.214922 May 25 16:38:49.963426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 687/1351] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:49 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:49.967884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2a13a308-8d7b-4310-beb8-3a8a2d12600d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.217070 May 25 16:38:49.976128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 665/1352] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:49 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:49.976320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5ab23c6b-11f0-4d09-8bf8-fd24cc2c5d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:49.976439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4c56d29f-bb54-4279-8aae-906f4c6fa599 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:49.976562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5ab23c6b-11f0-4d09-8bf8-fd24cc2c5d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:49.976672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4c56d29f-bb54-4279-8aae-906f4c6fa599 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:49.979730 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5ab23c6b-11f0-4d09-8bf8-fd24cc2c5d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:49.980178 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5ab23c6b-11f0-4d09-8bf8-fd24cc2c5d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:49.980936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5ab23c6b-11f0-4d09-8bf8-fd24cc2c5d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:49.981365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5ab23c6b-11f0-4d09-8bf8-fd24cc2c5d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:49.981528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c56d29f-bb54-4279-8aae-906f4c6fa599 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:49.981904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c56d29f-bb54-4279-8aae-906f4c6fa599 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:49.982750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c56d29f-bb54-4279-8aae-906f4c6fa599 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:49.983187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c56d29f-bb54-4279-8aae-906f4c6fa599 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:49.995425 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:50.011275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5ab23c6b-11f0-4d09-8bf8-fd24cc2c5d12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042885 May 25 16:38:50.011747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 688/1353] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:49 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:50.016289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:50.016888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:50.022883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:50.023373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:50.024124 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:50.024532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:50.047868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4c56d29f-bb54-4279-8aae-906f4c6fa599 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:50.056026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4c56d29f-bb54-4279-8aae-906f4c6fa599 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.086454 May 25 16:38:50.056598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 666/1354] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:49 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:50.157033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:50.166190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:50.167729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:50.168912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8ced43b8-dba9-4590-9471-e44ca5f3db54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.156222 May 25 16:38:50.169469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 689/1355] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:50 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:50.984345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c52aac2c-245d-4697-b4d6-7575b37b1ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:50.985314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c52aac2c-245d-4697-b4d6-7575b37b1ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:50.988714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e913ec28-5c56-4f1a-86f0-83c4f40f473a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:50.989661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e913ec28-5c56-4f1a-86f0-83c4f40f473a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:50.995569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c52aac2c-245d-4697-b4d6-7575b37b1ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:50.996223 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c52aac2c-245d-4697-b4d6-7575b37b1ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:50.997419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c52aac2c-245d-4697-b4d6-7575b37b1ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:50.998073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c52aac2c-245d-4697-b4d6-7575b37b1ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:51.000057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e913ec28-5c56-4f1a-86f0-83c4f40f473a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:51.000744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e913ec28-5c56-4f1a-86f0-83c4f40f473a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:51.002164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e913ec28-5c56-4f1a-86f0-83c4f40f473a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:51.002841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e913ec28-5c56-4f1a-86f0-83c4f40f473a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:51.190865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e913ec28-5c56-4f1a-86f0-83c4f40f473a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:51.193699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c52aac2c-245d-4697-b4d6-7575b37b1ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:51.199762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e913ec28-5c56-4f1a-86f0-83c4f40f473a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.216942 May 25 16:38:51.200351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 690/1356] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:50 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:51.201748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c52aac2c-245d-4697-b4d6-7575b37b1ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.223661 May 25 16:38:51.202306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 667/1357] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:50 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:51.205238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1be07f42-a664-4a20-868a-595b78b095d7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:51.206230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1be07f42-a664-4a20-868a-595b78b095d7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:51.207443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-83902e2b-c5a2-4653-a0f6-9f562af138a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:51.208201 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-83902e2b-c5a2-4653-a0f6-9f562af138a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:51.213113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1be07f42-a664-4a20-868a-595b78b095d7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:51.213608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1be07f42-a664-4a20-868a-595b78b095d7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:51.214383 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1be07f42-a664-4a20-868a-595b78b095d7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:51.214824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1be07f42-a664-4a20-868a-595b78b095d7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:51.216633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83902e2b-c5a2-4653-a0f6-9f562af138a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:51.217078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83902e2b-c5a2-4653-a0f6-9f562af138a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:51.217825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83902e2b-c5a2-4653-a0f6-9f562af138a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:51.218249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83902e2b-c5a2-4653-a0f6-9f562af138a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:51.244526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1be07f42-a664-4a20-868a-595b78b095d7 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043104 May 25 16:38:51.245178 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 691/1358] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:51 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:51.250084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:51.250677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:51.256859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:51.257312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:51.258084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:51.258521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:51.280285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-83902e2b-c5a2-4653-a0f6-9f562af138a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:51.288876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-83902e2b-c5a2-4653-a0f6-9f562af138a5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085263 May 25 16:38:51.289450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 668/1359] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:51 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:51.389584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:51.398573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:51.400121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:51.401303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a2cd8d72-b43a-4599-94d9-ff49db234c12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.155043 May 25 16:38:51.401851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 692/1360] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:52.219803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-67e6acda-11e4-4f59-afcc-52c3fce4c087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:52.220312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-666041b2-66eb-4f48-a3ba-b4ae62a29194 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:52.220692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-67e6acda-11e4-4f59-afcc-52c3fce4c087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:52.220800 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-666041b2-66eb-4f48-a3ba-b4ae62a29194 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:52.227258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67e6acda-11e4-4f59-afcc-52c3fce4c087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.227734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67e6acda-11e4-4f59-afcc-52c3fce4c087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.227973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-666041b2-66eb-4f48-a3ba-b4ae62a29194 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.228618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67e6acda-11e4-4f59-afcc-52c3fce4c087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.228763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-666041b2-66eb-4f48-a3ba-b4ae62a29194 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.229057 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-67e6acda-11e4-4f59-afcc-52c3fce4c087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.230028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-666041b2-66eb-4f48-a3ba-b4ae62a29194 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.230734 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-666041b2-66eb-4f48-a3ba-b4ae62a29194 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.430255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-67e6acda-11e4-4f59-afcc-52c3fce4c087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:52.433662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-666041b2-66eb-4f48-a3ba-b4ae62a29194 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:52.438669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-67e6acda-11e4-4f59-afcc-52c3fce4c087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222867 May 25 16:38:52.439443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 669/1361] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:52 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:52.443925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-86b89285-cc69-462f-8775-9a4123046134 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:52.444894 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-86b89285-cc69-462f-8775-9a4123046134 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:52.450382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-666041b2-66eb-4f48-a3ba-b4ae62a29194 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.234387 May 25 16:38:52.450967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86b89285-cc69-462f-8775-9a4123046134 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.451949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 693/1362] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:52 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:38:52.452093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86b89285-cc69-462f-8775-9a4123046134 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.452833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86b89285-cc69-462f-8775-9a4123046134 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.453271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86b89285-cc69-462f-8775-9a4123046134 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.455803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0f11cd91-4f37-47a0-b91c-df021c266a91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:52.456411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0f11cd91-4f37-47a0-b91c-df021c266a91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:52.462331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0f11cd91-4f37-47a0-b91c-df021c266a91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.462784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0f11cd91-4f37-47a0-b91c-df021c266a91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.463582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0f11cd91-4f37-47a0-b91c-df021c266a91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.464014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0f11cd91-4f37-47a0-b91c-df021c266a91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.483640 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-86b89285-cc69-462f-8775-9a4123046134 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043277 May 25 16:38:52.487779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 670/1363] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:52 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:52.488600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:52.489198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:52.495129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.495600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.496356 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:52.496784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:52.527850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0f11cd91-4f37-47a0-b91c-df021c266a91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:52.536298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0f11cd91-4f37-47a0-b91c-df021c266a91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084195 May 25 16:38:52.536929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 694/1364] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:52 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:52.635350 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:52.649191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:52.651493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:52.653251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-930ec5d5-576b-4d17-aa38-e5da27331ebc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.167850 May 25 16:38:52.653938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 671/1365] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:53.460533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1740f56f-7274-4759-a891-48e879236dd0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:53.461363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1740f56f-7274-4759-a891-48e879236dd0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:53.471714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ec2b2425-7d07-4af9-bff5-feba078fa751 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:53.472488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1740f56f-7274-4759-a891-48e879236dd0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.472709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ec2b2425-7d07-4af9-bff5-feba078fa751 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:53.473083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1740f56f-7274-4759-a891-48e879236dd0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.474477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1740f56f-7274-4759-a891-48e879236dd0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.474940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1740f56f-7274-4759-a891-48e879236dd0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.480345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ec2b2425-7d07-4af9-bff5-feba078fa751 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.480802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ec2b2425-7d07-4af9-bff5-feba078fa751 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.481573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ec2b2425-7d07-4af9-bff5-feba078fa751 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.482008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ec2b2425-7d07-4af9-bff5-feba078fa751 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.631419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:38:53.631669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:38:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a (ip 10.222.0.45) !!! May 25 16:38:53.672399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1740f56f-7274-4759-a891-48e879236dd0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:53.675701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ec2b2425-7d07-4af9-bff5-feba078fa751 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:53.680733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1740f56f-7274-4759-a891-48e879236dd0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.226931 May 25 16:38:53.681314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 695/1366] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:53 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:53.684051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ec2b2425-7d07-4af9-bff5-feba078fa751 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.218957 May 25 16:38:53.684751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 672/1367] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:53 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:53.685992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-45b46bbd-4d0f-4ebd-ad63-fe378331fd37 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:53.686980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-45b46bbd-4d0f-4ebd-ad63-fe378331fd37 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:53.690358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d6ee3157-3bc6-4db4-b788-a0eacc2d0be0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:53.691222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d6ee3157-3bc6-4db4-b788-a0eacc2d0be0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:53.693100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45b46bbd-4d0f-4ebd-ad63-fe378331fd37 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.693550 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45b46bbd-4d0f-4ebd-ad63-fe378331fd37 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.694317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45b46bbd-4d0f-4ebd-ad63-fe378331fd37 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.694756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45b46bbd-4d0f-4ebd-ad63-fe378331fd37 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.697428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6ee3157-3bc6-4db4-b788-a0eacc2d0be0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.697869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6ee3157-3bc6-4db4-b788-a0eacc2d0be0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.698621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6ee3157-3bc6-4db4-b788-a0eacc2d0be0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.699051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d6ee3157-3bc6-4db4-b788-a0eacc2d0be0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.724326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-45b46bbd-4d0f-4ebd-ad63-fe378331fd37 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041838 May 25 16:38:53.724811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 696/1368] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:53 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:53.729156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:53.729757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:53.735690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.736140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.736935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:53.737370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:53.759257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d6ee3157-3bc6-4db4-b788-a0eacc2d0be0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:53.767531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d6ee3157-3bc6-4db4-b788-a0eacc2d0be0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.081083 May 25 16:38:53.768137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 673/1369] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:53 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:53.868275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:53.881320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:53.882830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:53.884031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e732e9a3-7b06-4918-8fa8-b5a3794d4bdc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.158321 May 25 16:38:53.884608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 697/1370] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:53 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:54.699347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e7d4ef65-4c5c-4f33-9add-61d0a1d5729c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:54.700014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e7d4ef65-4c5c-4f33-9add-61d0a1d5729c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:54.701407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-84f93e22-4bf1-4f91-b84d-e02fe0856df1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:54.702047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-84f93e22-4bf1-4f91-b84d-e02fe0856df1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:54.707551 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7d4ef65-4c5c-4f33-9add-61d0a1d5729c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.707902 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7d4ef65-4c5c-4f33-9add-61d0a1d5729c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.708697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7d4ef65-4c5c-4f33-9add-61d0a1d5729c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.709145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7d4ef65-4c5c-4f33-9add-61d0a1d5729c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.709301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84f93e22-4bf1-4f91-b84d-e02fe0856df1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.709728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84f93e22-4bf1-4f91-b84d-e02fe0856df1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.710514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84f93e22-4bf1-4f91-b84d-e02fe0856df1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.710964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84f93e22-4bf1-4f91-b84d-e02fe0856df1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.904669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e7d4ef65-4c5c-4f33-9add-61d0a1d5729c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:54.909172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-84f93e22-4bf1-4f91-b84d-e02fe0856df1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:54.915682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e7d4ef65-4c5c-4f33-9add-61d0a1d5729c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220393 May 25 16:38:54.916465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 674/1371] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:54 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:54.918654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-84f93e22-4bf1-4f91-b84d-e02fe0856df1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.221189 May 25 16:38:54.928246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 698/1372] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:54 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:54.928461 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-091792b0-c2cc-433a-b534-2b6365fd4945 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:54.928461 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-091792b0-c2cc-433a-b534-2b6365fd4945 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:54.928669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d3da47fe-da51-4d75-8127-4db41b79b105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:54.928669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d3da47fe-da51-4d75-8127-4db41b79b105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:54.928850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-091792b0-c2cc-433a-b534-2b6365fd4945 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.928850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-091792b0-c2cc-433a-b534-2b6365fd4945 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.929065 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-091792b0-c2cc-433a-b534-2b6365fd4945 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.929463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-091792b0-c2cc-433a-b534-2b6365fd4945 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.930181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d3da47fe-da51-4d75-8127-4db41b79b105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.930633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d3da47fe-da51-4d75-8127-4db41b79b105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.931421 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d3da47fe-da51-4d75-8127-4db41b79b105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.931857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d3da47fe-da51-4d75-8127-4db41b79b105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.961759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-091792b0-c2cc-433a-b534-2b6365fd4945 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044391 May 25 16:38:54.962242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 675/1373] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:54 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:54.967363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:54.968017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:54.974180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.974626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.975445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:54.975886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:54.995566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d3da47fe-da51-4d75-8127-4db41b79b105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:55.003550 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d3da47fe-da51-4d75-8127-4db41b79b105 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083350 May 25 16:38:55.004318 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 699/1374] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:54 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:55.107730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:55.116594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:55.118090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:55.119281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c78619d8-24cc-4708-b419-0d76db517c3f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.156067 May 25 16:38:55.119838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 676/1375] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:55.937848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d449df4d-845a-410b-969e-055d245c0337 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:55.938900 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d449df4d-845a-410b-969e-055d245c0337 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:55.939520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-70a0ccc7-a4b1-46f1-bf70-6992244194b2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:55.940519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-70a0ccc7-a4b1-46f1-bf70-6992244194b2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:55.949986 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d449df4d-845a-410b-969e-055d245c0337 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:55.950742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d449df4d-845a-410b-969e-055d245c0337 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:55.951775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70a0ccc7-a4b1-46f1-bf70-6992244194b2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:55.952020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d449df4d-845a-410b-969e-055d245c0337 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:55.952513 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70a0ccc7-a4b1-46f1-bf70-6992244194b2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:55.952844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d449df4d-845a-410b-969e-055d245c0337 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:55.953852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70a0ccc7-a4b1-46f1-bf70-6992244194b2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:55.954571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70a0ccc7-a4b1-46f1-bf70-6992244194b2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:56.145743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d449df4d-845a-410b-969e-055d245c0337 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:56.150401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-70a0ccc7-a4b1-46f1-bf70-6992244194b2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:56.154086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d449df4d-845a-410b-969e-055d245c0337 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222807 May 25 16:38:56.154667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 700/1376] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:55 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:56.158523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-70a0ccc7-a4b1-46f1-bf70-6992244194b2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.225460 May 25 16:38:56.159208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 677/1377] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:55 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:56.159377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ce1e4d69-87df-4e48-963c-121514dfd537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:56.160385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ce1e4d69-87df-4e48-963c-121514dfd537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:56.167877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e2ad0c6f-3fd2-4e37-8ee7-986955561843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:56.168507 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e2ad0c6f-3fd2-4e37-8ee7-986955561843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:56.169468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce1e4d69-87df-4e48-963c-121514dfd537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:56.169916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce1e4d69-87df-4e48-963c-121514dfd537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:56.170667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce1e4d69-87df-4e48-963c-121514dfd537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:56.171318 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce1e4d69-87df-4e48-963c-121514dfd537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:56.174357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2ad0c6f-3fd2-4e37-8ee7-986955561843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:56.174807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2ad0c6f-3fd2-4e37-8ee7-986955561843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:56.175572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2ad0c6f-3fd2-4e37-8ee7-986955561843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:56.176027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2ad0c6f-3fd2-4e37-8ee7-986955561843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:56.178605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:56.201371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ce1e4d69-87df-4e48-963c-121514dfd537 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045554 May 25 16:38:56.201866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 701/1378] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:56 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:56.206266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:56.207294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:56.213022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:56.213468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:56.214290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:56.214723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:56.241884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e2ad0c6f-3fd2-4e37-8ee7-986955561843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:56.250004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e2ad0c6f-3fd2-4e37-8ee7-986955561843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.086025 May 25 16:38:56.250574 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 678/1379] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:56 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:56.345785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:56.356056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:56.357590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:56.358736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-01d6e921-4ea0-44d2-a147-ccd8a4f57f2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.155919 May 25 16:38:56.359322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 702/1380] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:56 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:57.176145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c5c4d980-5d14-4e75-8d23-2a36ae4bafc2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:57.177278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c5c4d980-5d14-4e75-8d23-2a36ae4bafc2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:57.180576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-40c02ab4-885d-4fb3-b6ba-6ff87edfc367 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:57.181655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-40c02ab4-885d-4fb3-b6ba-6ff87edfc367 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:57.188080 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c5c4d980-5d14-4e75-8d23-2a36ae4bafc2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.188795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c5c4d980-5d14-4e75-8d23-2a36ae4bafc2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.190028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c5c4d980-5d14-4e75-8d23-2a36ae4bafc2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.190829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c5c4d980-5d14-4e75-8d23-2a36ae4bafc2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.192977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40c02ab4-885d-4fb3-b6ba-6ff87edfc367 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.193441 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40c02ab4-885d-4fb3-b6ba-6ff87edfc367 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.194216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40c02ab4-885d-4fb3-b6ba-6ff87edfc367 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.194646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40c02ab4-885d-4fb3-b6ba-6ff87edfc367 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.388224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c5c4d980-5d14-4e75-8d23-2a36ae4bafc2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:57.391548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-40c02ab4-885d-4fb3-b6ba-6ff87edfc367 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:57.397059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c5c4d980-5d14-4e75-8d23-2a36ae4bafc2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.227476 May 25 16:38:57.397981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 679/1381] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:57 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:57.400105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-40c02ab4-885d-4fb3-b6ba-6ff87edfc367 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.226541 May 25 16:38:57.408773 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 703/1382] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:57 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:57.408984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5232a422-08c1-4008-af23-5f08dc2a7024 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:57.408984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5232a422-08c1-4008-af23-5f08dc2a7024 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:57.409167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-84bdeb14-942f-4a19-91b2-0a955b17e520 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:57.409167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-84bdeb14-942f-4a19-91b2-0a955b17e520 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:57.410298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5232a422-08c1-4008-af23-5f08dc2a7024 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.410751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5232a422-08c1-4008-af23-5f08dc2a7024 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.411579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5232a422-08c1-4008-af23-5f08dc2a7024 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.412035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5232a422-08c1-4008-af23-5f08dc2a7024 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.412870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84bdeb14-942f-4a19-91b2-0a955b17e520 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.413349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84bdeb14-942f-4a19-91b2-0a955b17e520 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.414095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84bdeb14-942f-4a19-91b2-0a955b17e520 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.414863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-84bdeb14-942f-4a19-91b2-0a955b17e520 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.444632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5232a422-08c1-4008-af23-5f08dc2a7024 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045564 May 25 16:38:57.445318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 680/1383] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:57 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:57.450114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:57.450800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:57.457249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.457725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.458668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:57.459112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:57.476733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-84bdeb14-942f-4a19-91b2-0a955b17e520 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:57.487487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-84bdeb14-942f-4a19-91b2-0a955b17e520 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084128 May 25 16:38:57.494700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 704/1384] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:57 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:57.595527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:57.608142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:57.610368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:57.612137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cc956048-3e92-4ad6-94be-20a895790d5e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.165607 May 25 16:38:57.612910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 681/1385] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:58.417306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8e6f1d94-6c22-4845-8c15-1dc31fd02bcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:58.418002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8e6f1d94-6c22-4845-8c15-1dc31fd02bcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:58.420360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9a0c815a-ef4e-4d0a-a1fa-d19f531beda6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:58.421517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9a0c815a-ef4e-4d0a-a1fa-d19f531beda6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:58.424706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e6f1d94-6c22-4845-8c15-1dc31fd02bcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.425149 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e6f1d94-6c22-4845-8c15-1dc31fd02bcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.425904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e6f1d94-6c22-4845-8c15-1dc31fd02bcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.426321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e6f1d94-6c22-4845-8c15-1dc31fd02bcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.427681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9a0c815a-ef4e-4d0a-a1fa-d19f531beda6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.428146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9a0c815a-ef4e-4d0a-a1fa-d19f531beda6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.428907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9a0c815a-ef4e-4d0a-a1fa-d19f531beda6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.429027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:58.429336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9a0c815a-ef4e-4d0a-a1fa-d19f531beda6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.618736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8e6f1d94-6c22-4845-8c15-1dc31fd02bcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:58.621721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9a0c815a-ef4e-4d0a-a1fa-d19f531beda6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:58.627566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8e6f1d94-6c22-4845-8c15-1dc31fd02bcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.215079 May 25 16:38:58.628126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 705/1386] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:58 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:58.630342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9a0c815a-ef4e-4d0a-a1fa-d19f531beda6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.214778 May 25 16:38:58.631121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 682/1387] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:58 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:58.633007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1591f6d3-e9c5-4f30-b580-71146cfbd4ae tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:58.634002 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1591f6d3-e9c5-4f30-b580-71146cfbd4ae tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:58.640327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1591f6d3-e9c5-4f30-b580-71146cfbd4ae tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.640734 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1591f6d3-e9c5-4f30-b580-71146cfbd4ae tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.641530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1591f6d3-e9c5-4f30-b580-71146cfbd4ae tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.641961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1591f6d3-e9c5-4f30-b580-71146cfbd4ae tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.643789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9cb23221-c6f7-441a-adcd-f6f30b3f6014 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:58.644390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9cb23221-c6f7-441a-adcd-f6f30b3f6014 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:58.650367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cb23221-c6f7-441a-adcd-f6f30b3f6014 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.650818 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cb23221-c6f7-441a-adcd-f6f30b3f6014 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.651603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cb23221-c6f7-441a-adcd-f6f30b3f6014 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.652035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9cb23221-c6f7-441a-adcd-f6f30b3f6014 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.671434 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1591f6d3-e9c5-4f30-b580-71146cfbd4ae tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042145 May 25 16:38:58.671892 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 706/1388] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:58 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:58.676451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:58.677056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:58.683271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.683719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.684478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:58.684911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:58.715809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9cb23221-c6f7-441a-adcd-f6f30b3f6014 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:58.724298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9cb23221-c6f7-441a-adcd-f6f30b3f6014 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083957 May 25 16:38:58.725096 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 683/1389] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:58 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:58.815232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:58.824241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:58.825735 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:58.826901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b9b2cd1b-bed9-484e-9375-1f1c9cb616f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.154099 May 25 16:38:58.827500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 707/1390] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:58 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:38:59.647910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fc7842cf-2ccd-4d74-b37b-5adc8d82f9ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:59.648521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fc7842cf-2ccd-4d74-b37b-5adc8d82f9ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:59.649270 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-220940fa-f00d-4594-ba40-0474f2cdc4f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:59.649882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-220940fa-f00d-4594-ba40-0474f2cdc4f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:59.654965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc7842cf-2ccd-4d74-b37b-5adc8d82f9ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.655441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc7842cf-2ccd-4d74-b37b-5adc8d82f9ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.656205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc7842cf-2ccd-4d74-b37b-5adc8d82f9ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.656415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-220940fa-f00d-4594-ba40-0474f2cdc4f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.656638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc7842cf-2ccd-4d74-b37b-5adc8d82f9ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.657140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-220940fa-f00d-4594-ba40-0474f2cdc4f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.657894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-220940fa-f00d-4594-ba40-0474f2cdc4f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.658326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-220940fa-f00d-4594-ba40-0474f2cdc4f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.856060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fc7842cf-2ccd-4d74-b37b-5adc8d82f9ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:59.859010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-220940fa-f00d-4594-ba40-0474f2cdc4f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:59.864817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fc7842cf-2ccd-4d74-b37b-5adc8d82f9ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222136 May 25 16:38:59.865459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 684/1391] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:59 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:59.868927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-220940fa-f00d-4594-ba40-0474f2cdc4f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.224123 May 25 16:38:59.869687 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 708/1392] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:59 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:38:59.870497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-105b1ff9-1d3e-47d6-b4dd-270b1f53cf1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:59.871568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-105b1ff9-1d3e-47d6-b4dd-270b1f53cf1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:59.877629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-105b1ff9-1d3e-47d6-b4dd-270b1f53cf1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.878089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-105b1ff9-1d3e-47d6-b4dd-270b1f53cf1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.878877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-105b1ff9-1d3e-47d6-b4dd-270b1f53cf1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.879338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-105b1ff9-1d3e-47d6-b4dd-270b1f53cf1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.883821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-61cf9b17-bbe2-4711-a41f-1b2a14bca0ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:59.884438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-61cf9b17-bbe2-4711-a41f-1b2a14bca0ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:59.890768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-61cf9b17-bbe2-4711-a41f-1b2a14bca0ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.891335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-61cf9b17-bbe2-4711-a41f-1b2a14bca0ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.892145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-61cf9b17-bbe2-4711-a41f-1b2a14bca0ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.892599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-61cf9b17-bbe2-4711-a41f-1b2a14bca0ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.910224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-105b1ff9-1d3e-47d6-b4dd-270b1f53cf1b tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043558 May 25 16:38:59.910977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 685/1393] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:38:59 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:38:59.915820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:38:59.916497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:38:59.922489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.922979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.923905 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:38:59.924363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:38:59.956349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-61cf9b17-bbe2-4711-a41f-1b2a14bca0ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:38:59.964843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-61cf9b17-bbe2-4711-a41f-1b2a14bca0ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084869 May 25 16:38:59.965428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 709/1394] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:38:59 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:00.077642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:00.086497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:00.088034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:00.089200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fa1a9fcd-a60d-405b-93b7-1bc9119fc102 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.177454 May 25 16:39:00.089760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 686/1395] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:38:59 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:00.884222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-14ee7ece-c5df-4add-8c63-bf0a541e2e77 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:00.884909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-14ee7ece-c5df-4add-8c63-bf0a541e2e77 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:00.886581 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7bc00c14-a4dd-46ba-9193-ef759d362d8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:00.887228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7bc00c14-a4dd-46ba-9193-ef759d362d8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:00.891675 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-14ee7ece-c5df-4add-8c63-bf0a541e2e77 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:00.892114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-14ee7ece-c5df-4add-8c63-bf0a541e2e77 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:00.892861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-14ee7ece-c5df-4add-8c63-bf0a541e2e77 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:00.893286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-14ee7ece-c5df-4add-8c63-bf0a541e2e77 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:00.893990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7bc00c14-a4dd-46ba-9193-ef759d362d8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:00.894467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7bc00c14-a4dd-46ba-9193-ef759d362d8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:00.895284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7bc00c14-a4dd-46ba-9193-ef759d362d8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:00.895716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7bc00c14-a4dd-46ba-9193-ef759d362d8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:01.097098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-14ee7ece-c5df-4add-8c63-bf0a541e2e77 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:01.100465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7bc00c14-a4dd-46ba-9193-ef759d362d8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:01.105339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-14ee7ece-c5df-4add-8c63-bf0a541e2e77 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.225894 May 25 16:39:01.106121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 710/1396] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:00 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:01.108789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7bc00c14-a4dd-46ba-9193-ef759d362d8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.226158 May 25 16:39:01.109355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 687/1397] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:00 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:01.111477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-87b90bf7-fa77-4189-9758-26e4d3229c33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:01.112453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-87b90bf7-fa77-4189-9758-26e4d3229c33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:01.114746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2e89ae7e-84f1-478c-9109-721b3a964bac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:01.115531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2e89ae7e-84f1-478c-9109-721b3a964bac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:01.120566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87b90bf7-fa77-4189-9758-26e4d3229c33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:01.120980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87b90bf7-fa77-4189-9758-26e4d3229c33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:01.121945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87b90bf7-fa77-4189-9758-26e4d3229c33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:01.122347 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-87b90bf7-fa77-4189-9758-26e4d3229c33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:01.126962 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e89ae7e-84f1-478c-9109-721b3a964bac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:01.127524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e89ae7e-84f1-478c-9109-721b3a964bac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:01.130680 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e89ae7e-84f1-478c-9109-721b3a964bac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:01.131144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2e89ae7e-84f1-478c-9109-721b3a964bac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:01.155528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-87b90bf7-fa77-4189-9758-26e4d3229c33 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048048 May 25 16:39:01.155994 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 711/1398] 10.222.0.45 () {58 vars in 1301 bytes} [Wed May 25 16:39:01 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:39:01.160656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:01.161979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:01.168272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:01.168725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:01.169488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:01.169923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:01.196273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2e89ae7e-84f1-478c-9109-721b3a964bac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:01.204285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2e89ae7e-84f1-478c-9109-721b3a964bac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093548 May 25 16:39:01.204855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 688/1399] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:01 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:01.309788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:01.318944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:01.320485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:01.321647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b06eb05c-4e0c-4c61-b3f2-2167465d1803 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.164707 May 25 16:39:01.322217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 712/1400] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:01 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:02.129194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-416c29a2-d1ba-43b6-94f4-2d6eb549f4bd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:02.129910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4868a41d-3d29-4abe-8358-0f2c8a497293 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:02.130510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-416c29a2-d1ba-43b6-94f4-2d6eb549f4bd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:02.130612 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4868a41d-3d29-4abe-8358-0f2c8a497293 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:02.137325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-416c29a2-d1ba-43b6-94f4-2d6eb549f4bd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.137528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4868a41d-3d29-4abe-8358-0f2c8a497293 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.137898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-416c29a2-d1ba-43b6-94f4-2d6eb549f4bd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.138043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4868a41d-3d29-4abe-8358-0f2c8a497293 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.138656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-416c29a2-d1ba-43b6-94f4-2d6eb549f4bd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.138802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4868a41d-3d29-4abe-8358-0f2c8a497293 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.139082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-416c29a2-d1ba-43b6-94f4-2d6eb549f4bd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.139281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4868a41d-3d29-4abe-8358-0f2c8a497293 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.332262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-416c29a2-d1ba-43b6-94f4-2d6eb549f4bd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:02.335942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4868a41d-3d29-4abe-8358-0f2c8a497293 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:02.340877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-416c29a2-d1ba-43b6-94f4-2d6eb549f4bd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.216749 May 25 16:39:02.341456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 689/1401] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:02 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:02.345083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4868a41d-3d29-4abe-8358-0f2c8a497293 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.220440 May 25 16:39:02.354856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 713/1402] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:02 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:02.355049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-256108b2-561a-4309-ab8d-4caa08d02cbf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:39:02.355049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-256108b2-561a-4309-ab8d-4caa08d02cbf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:02.355402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dd9ecb0f-4945-4487-a2a3-d30aac0d4291 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:02.355402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dd9ecb0f-4945-4487-a2a3-d30aac0d4291 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:02.355590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-256108b2-561a-4309-ab8d-4caa08d02cbf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.355590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-256108b2-561a-4309-ab8d-4caa08d02cbf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.355590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-256108b2-561a-4309-ab8d-4caa08d02cbf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.355590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-256108b2-561a-4309-ab8d-4caa08d02cbf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.358523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd9ecb0f-4945-4487-a2a3-d30aac0d4291 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.359021 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd9ecb0f-4945-4487-a2a3-d30aac0d4291 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.359872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd9ecb0f-4945-4487-a2a3-d30aac0d4291 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.360374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dd9ecb0f-4945-4487-a2a3-d30aac0d4291 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.426909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dd9ecb0f-4945-4487-a2a3-d30aac0d4291 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:02.434444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-256108b2-561a-4309-ab8d-4caa08d02cbf tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "POST /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/action" status: 200 len: 118995 microversion: 2.1 time: 0.091784 May 25 16:39:02.435045 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dd9ecb0f-4945-4487-a2a3-d30aac0d4291 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088012 May 25 16:39:02.435245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 690/1403] 10.222.0.45 () {60 vars in 1277 bytes} [Wed May 25 16:39:02 2022] POST /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a/action => generated 118995 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:02.435944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 714/1404] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:02 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:02.440506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:02.441200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:02.448832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.449302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.449906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:02.451148 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:02.598317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:02.607379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:02.608877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:02.610038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9b75edd5-a2e9-4350-bb80-558f8c511272 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.173597 May 25 16:39:02.610590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 691/1405] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:02 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:03.362143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0c212f9c-5d1b-4ba0-b55e-965160573ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:03.362995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0c212f9c-5d1b-4ba0-b55e-965160573ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:03.365650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e414383d-e3e5-47fa-8c68-0aab537dd05b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:03.366341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e414383d-e3e5-47fa-8c68-0aab537dd05b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:03.370673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c212f9c-5d1b-4ba0-b55e-965160573ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:03.371114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c212f9c-5d1b-4ba0-b55e-965160573ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:03.371905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c212f9c-5d1b-4ba0-b55e-965160573ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:03.372336 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c212f9c-5d1b-4ba0-b55e-965160573ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:03.373106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e414383d-e3e5-47fa-8c68-0aab537dd05b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:03.373558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e414383d-e3e5-47fa-8c68-0aab537dd05b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:03.374317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e414383d-e3e5-47fa-8c68-0aab537dd05b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:03.374748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e414383d-e3e5-47fa-8c68-0aab537dd05b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:03.571919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0c212f9c-5d1b-4ba0-b55e-965160573ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:03.578044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e414383d-e3e5-47fa-8c68-0aab537dd05b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:03.584802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0c212f9c-5d1b-4ba0-b55e-965160573ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.229388 May 25 16:39:03.585362 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 715/1406] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:03 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:03.589532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e414383d-e3e5-47fa-8c68-0aab537dd05b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.229989 May 25 16:39:03.589772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b7141e52-c7fa-4da2-9c2b-63eeebc50cd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:03.590504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 692/1407] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:03 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:03.590637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b7141e52-c7fa-4da2-9c2b-63eeebc50cd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:03.596704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b7141e52-c7fa-4da2-9c2b-63eeebc50cd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:03.597164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b7141e52-c7fa-4da2-9c2b-63eeebc50cd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:03.597937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b7141e52-c7fa-4da2-9c2b-63eeebc50cd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:03.598376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b7141e52-c7fa-4da2-9c2b-63eeebc50cd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:03.628163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:03.628768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:03.635567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:03.636090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:03.636925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:03.637417 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:03.659118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b7141e52-c7fa-4da2-9c2b-63eeebc50cd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:03.668040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b7141e52-c7fa-4da2-9c2b-63eeebc50cd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.081716 May 25 16:39:03.668611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 716/1408] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:03 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:03.769161 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:03.778313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:03.779829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:03.781101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-31f08f63-9966-40cf-99c3-da5d41feb4af tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.157077 May 25 16:39:03.781645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 693/1409] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:03 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:04.604897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-02dc5416-052e-46f6-ac00-06bfa7a0d4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:04.605621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-02dc5416-052e-46f6-ac00-06bfa7a0d4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:04.609395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1546669a-fcd9-465b-806c-fa6389937540 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:04.610274 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1546669a-fcd9-465b-806c-fa6389937540 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:04.613632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02dc5416-052e-46f6-ac00-06bfa7a0d4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:04.614164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02dc5416-052e-46f6-ac00-06bfa7a0d4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:04.615094 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02dc5416-052e-46f6-ac00-06bfa7a0d4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:04.615644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02dc5416-052e-46f6-ac00-06bfa7a0d4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:04.617865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1546669a-fcd9-465b-806c-fa6389937540 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:04.618395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1546669a-fcd9-465b-806c-fa6389937540 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:04.619330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1546669a-fcd9-465b-806c-fa6389937540 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:04.619844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1546669a-fcd9-465b-806c-fa6389937540 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:05.050927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-02dc5416-052e-46f6-ac00-06bfa7a0d4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:05.054293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1546669a-fcd9-465b-806c-fa6389937540 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:05.059341 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-02dc5416-052e-46f6-ac00-06bfa7a0d4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.460166 May 25 16:39:05.060408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 717/1410] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:04 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:05.063077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1546669a-fcd9-465b-806c-fa6389937540 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.458116 May 25 16:39:05.072636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-76693d7f-fedb-46d7-8897-08a30e9d527f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:05.072636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-76693d7f-fedb-46d7-8897-08a30e9d527f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:05.073158 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-76693d7f-fedb-46d7-8897-08a30e9d527f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:05.073388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 694/1411] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:04 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:05.073609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-76693d7f-fedb-46d7-8897-08a30e9d527f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:05.074373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-76693d7f-fedb-46d7-8897-08a30e9d527f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:05.074802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-76693d7f-fedb-46d7-8897-08a30e9d527f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:05.078177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:05.079203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:05.085136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:05.085588 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:05.086357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:05.086794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:05.090886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:05.144693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-76693d7f-fedb-46d7-8897-08a30e9d527f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:05.153186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-76693d7f-fedb-46d7-8897-08a30e9d527f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092316 May 25 16:39:05.153924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 718/1412] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:05 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:05.232658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:05.246292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:05.248734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:05.250345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-82a2310f-b8eb-4b3b-b785-2f98d2af9491 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.175907 May 25 16:39:05.251192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 695/1413] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:05 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:06.081471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5fcfe886-694e-47e1-89dd-5574a56acea7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:06.082493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5fcfe886-694e-47e1-89dd-5574a56acea7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:06.084850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a154a34f-90a9-404f-ba87-6bc81264a6a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:06.085853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a154a34f-90a9-404f-ba87-6bc81264a6a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:06.094349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5fcfe886-694e-47e1-89dd-5574a56acea7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:06.095047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5fcfe886-694e-47e1-89dd-5574a56acea7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:06.096360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5fcfe886-694e-47e1-89dd-5574a56acea7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:06.097092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5fcfe886-694e-47e1-89dd-5574a56acea7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:06.097330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a154a34f-90a9-404f-ba87-6bc81264a6a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:06.098047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a154a34f-90a9-404f-ba87-6bc81264a6a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:06.099357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a154a34f-90a9-404f-ba87-6bc81264a6a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:06.100152 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a154a34f-90a9-404f-ba87-6bc81264a6a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:06.290548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5fcfe886-694e-47e1-89dd-5574a56acea7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:06.294444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a154a34f-90a9-404f-ba87-6bc81264a6a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:06.298998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5fcfe886-694e-47e1-89dd-5574a56acea7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.224553 May 25 16:39:06.299607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 719/1414] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:06 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:06.302796 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a154a34f-90a9-404f-ba87-6bc81264a6a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.224711 May 25 16:39:06.311215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 696/1415] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:06 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:06.311423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-90f7d702-28dd-4b31-ae7d-5aebaace5175 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:06.311423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-90f7d702-28dd-4b31-ae7d-5aebaace5175 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:06.311594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:06.311594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:06.312359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90f7d702-28dd-4b31-ae7d-5aebaace5175 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:06.312836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90f7d702-28dd-4b31-ae7d-5aebaace5175 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:06.313603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90f7d702-28dd-4b31-ae7d-5aebaace5175 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:06.314038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90f7d702-28dd-4b31-ae7d-5aebaace5175 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:06.315971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:06.316489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:06.317350 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:06.317861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:06.400123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-90f7d702-28dd-4b31-ae7d-5aebaace5175 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:06.408708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-90f7d702-28dd-4b31-ae7d-5aebaace5175 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.108086 May 25 16:39:06.409636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 720/1416] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:06 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:06.468583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:06.477192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:06.478693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:06.479896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-224eaf8b-684c-491b-b5b4-ef8c1c069cce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.175080 May 25 16:39:06.480448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 697/1417] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:06 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:07.321104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-91e160cf-9f0c-45f3-a0c1-9e9fa3e06fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:07.322151 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-91e160cf-9f0c-45f3-a0c1-9e9fa3e06fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:07.324334 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-da6e1f4a-d23c-4b2f-b359-cafb860e41cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:07.325332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-da6e1f4a-d23c-4b2f-b359-cafb860e41cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:07.333257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-91e160cf-9f0c-45f3-a0c1-9e9fa3e06fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:07.333995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-91e160cf-9f0c-45f3-a0c1-9e9fa3e06fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:07.335019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-91e160cf-9f0c-45f3-a0c1-9e9fa3e06fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:07.335556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da6e1f4a-d23c-4b2f-b359-cafb860e41cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:07.335758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-91e160cf-9f0c-45f3-a0c1-9e9fa3e06fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:07.336198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da6e1f4a-d23c-4b2f-b359-cafb860e41cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:07.336782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da6e1f4a-d23c-4b2f-b359-cafb860e41cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:07.337213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da6e1f4a-d23c-4b2f-b359-cafb860e41cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:07.524913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-91e160cf-9f0c-45f3-a0c1-9e9fa3e06fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:07.527424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:07.528066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-da6e1f4a-d23c-4b2f-b359-cafb860e41cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:07.534850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-91e160cf-9f0c-45f3-a0c1-9e9fa3e06fed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220662 May 25 16:39:07.535484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 721/1418] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:07 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:07.536135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-da6e1f4a-d23c-4b2f-b359-cafb860e41cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.218163 May 25 16:39:07.544944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 698/1419] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:07 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:07.545142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-836ade3d-0978-47ca-bce0-5451ed6a9e56 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:07.545142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-836ade3d-0978-47ca-bce0-5451ed6a9e56 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:07.545328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:07.545328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:07.547183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-836ade3d-0978-47ca-bce0-5451ed6a9e56 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:07.547631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-836ade3d-0978-47ca-bce0-5451ed6a9e56 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:07.548398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-836ade3d-0978-47ca-bce0-5451ed6a9e56 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:07.548834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-836ade3d-0978-47ca-bce0-5451ed6a9e56 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:07.550042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:07.550493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:07.551276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:07.551709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:07.616590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-836ade3d-0978-47ca-bce0-5451ed6a9e56 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:07.624657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-836ade3d-0978-47ca-bce0-5451ed6a9e56 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088222 May 25 16:39:07.625238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 722/1420] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:07 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:07.699410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:07.708212 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:07.709747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:07.710904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a0fe856e-e670-4df2-baa8-bd5f41b48f41 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.172934 May 25 16:39:07.711494 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 699/1421] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:07 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:08.555623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f70053a5-7d41-4d16-8d70-399df24d647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:08.556218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d724da57-3c96-4dfa-a6d7-2a68fa4348d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:08.556615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f70053a5-7d41-4d16-8d70-399df24d647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:08.556714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d724da57-3c96-4dfa-a6d7-2a68fa4348d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:08.563403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f70053a5-7d41-4d16-8d70-399df24d647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:08.563675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d724da57-3c96-4dfa-a6d7-2a68fa4348d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:08.563979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f70053a5-7d41-4d16-8d70-399df24d647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:08.564126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d724da57-3c96-4dfa-a6d7-2a68fa4348d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:08.564767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f70053a5-7d41-4d16-8d70-399df24d647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:08.564886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d724da57-3c96-4dfa-a6d7-2a68fa4348d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:08.565201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f70053a5-7d41-4d16-8d70-399df24d647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:08.565311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d724da57-3c96-4dfa-a6d7-2a68fa4348d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:08.756470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f70053a5-7d41-4d16-8d70-399df24d647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:08.760090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d724da57-3c96-4dfa-a6d7-2a68fa4348d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:08.766870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f70053a5-7d41-4d16-8d70-399df24d647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.216402 May 25 16:39:08.767728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 723/1422] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:08 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:08.768514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d724da57-3c96-4dfa-a6d7-2a68fa4348d5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.217260 May 25 16:39:08.769385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 700/1423] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:08 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:08.772107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4fe23350-aa2d-4f89-aa18-326286ef60dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:08.772715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4fe23350-aa2d-4f89-aa18-326286ef60dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:08.778610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4fe23350-aa2d-4f89-aa18-326286ef60dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:08.779184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4fe23350-aa2d-4f89-aa18-326286ef60dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:08.779942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4fe23350-aa2d-4f89-aa18-326286ef60dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:08.780378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4fe23350-aa2d-4f89-aa18-326286ef60dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:08.782972 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:08.783593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:08.790085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:08.790526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:08.791312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:08.791736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:08.857108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4fe23350-aa2d-4f89-aa18-326286ef60dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:08.865305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4fe23350-aa2d-4f89-aa18-326286ef60dc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.096902 May 25 16:39:08.865917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 724/1424] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:08 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:08.945331 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:08.954530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:08.956062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:08.957257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-54f4343b-52ec-4117-b90f-2792e394c4d8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.178069 May 25 16:39:08.957809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 701/1425] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:08 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:09.788058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1b020015-ea5b-4d64-95ce-7c39a3070018 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:09.788646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1b020015-ea5b-4d64-95ce-7c39a3070018 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:09.788840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f9908488-bd0b-4950-ab45-63e8f0ab6c91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:09.789454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f9908488-bd0b-4950-ab45-63e8f0ab6c91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:09.792274 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:09.796898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b020015-ea5b-4d64-95ce-7c39a3070018 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:09.797390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b020015-ea5b-4d64-95ce-7c39a3070018 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:09.797854 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f9908488-bd0b-4950-ab45-63e8f0ab6c91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:09.798224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b020015-ea5b-4d64-95ce-7c39a3070018 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:09.798373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f9908488-bd0b-4950-ab45-63e8f0ab6c91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:09.798659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1b020015-ea5b-4d64-95ce-7c39a3070018 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:09.799042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f9908488-bd0b-4950-ab45-63e8f0ab6c91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:09.799499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f9908488-bd0b-4950-ab45-63e8f0ab6c91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:09.990995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1b020015-ea5b-4d64-95ce-7c39a3070018 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:09.998304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f9908488-bd0b-4950-ab45-63e8f0ab6c91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:10.000402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1b020015-ea5b-4d64-95ce-7c39a3070018 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.218677 May 25 16:39:10.000979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 725/1426] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:09 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:10.005353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-20fe0d71-e221-4c06-a31b-61b175d8b7fd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:10.006099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-20fe0d71-e221-4c06-a31b-61b175d8b7fd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:10.007172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f9908488-bd0b-4950-ab45-63e8f0ab6c91 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.223932 May 25 16:39:10.017304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 702/1427] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:09 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:10.017522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20fe0d71-e221-4c06-a31b-61b175d8b7fd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:10.017522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20fe0d71-e221-4c06-a31b-61b175d8b7fd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:10.017711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:10.017848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20fe0d71-e221-4c06-a31b-61b175d8b7fd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:10.017939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:10.018019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20fe0d71-e221-4c06-a31b-61b175d8b7fd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:10.020163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:10.020797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:10.022021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:10.022458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:10.099850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-20fe0d71-e221-4c06-a31b-61b175d8b7fd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:10.108101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-20fe0d71-e221-4c06-a31b-61b175d8b7fd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.106163 May 25 16:39:10.108671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 726/1428] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:10 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:10.170915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:10.181789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:10.183318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:10.184486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ed6624a8-b48e-41ef-b947-e87137303600 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.175429 May 25 16:39:10.185072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 703/1429] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:10 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:11.025695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-852bc819-9cd8-44ab-9662-da3b144a35c6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:11.027339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-852bc819-9cd8-44ab-9662-da3b144a35c6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:11.028153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ed4f35e2-36fa-4848-aa18-b7c24100a3cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:11.028908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ed4f35e2-36fa-4848-aa18-b7c24100a3cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:11.036987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed4f35e2-36fa-4848-aa18-b7c24100a3cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:11.037598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed4f35e2-36fa-4848-aa18-b7c24100a3cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:11.038536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed4f35e2-36fa-4848-aa18-b7c24100a3cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:11.038673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-852bc819-9cd8-44ab-9662-da3b144a35c6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:11.039066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed4f35e2-36fa-4848-aa18-b7c24100a3cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:11.039241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-852bc819-9cd8-44ab-9662-da3b144a35c6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:11.040610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-852bc819-9cd8-44ab-9662-da3b144a35c6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:11.041213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-852bc819-9cd8-44ab-9662-da3b144a35c6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:11.235847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ed4f35e2-36fa-4848-aa18-b7c24100a3cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:11.239101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-852bc819-9cd8-44ab-9662-da3b144a35c6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:11.244001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ed4f35e2-36fa-4848-aa18-b7c24100a3cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.221199 May 25 16:39:11.244655 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 704/1430] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:11 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:11.247368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-852bc819-9cd8-44ab-9662-da3b144a35c6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.231437 May 25 16:39:11.248100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 727/1431] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:11 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:11.249547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b439f114-67a8-4a47-b929-450df95233d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:11.250208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b439f114-67a8-4a47-b929-450df95233d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:11.252345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:11.252972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:11.256348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b439f114-67a8-4a47-b929-450df95233d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:11.256900 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b439f114-67a8-4a47-b929-450df95233d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:11.257722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b439f114-67a8-4a47-b929-450df95233d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:11.258277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b439f114-67a8-4a47-b929-450df95233d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:11.259554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:11.259995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:11.260750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:11.261173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:11.336543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b439f114-67a8-4a47-b929-450df95233d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:11.344735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b439f114-67a8-4a47-b929-450df95233d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.099057 May 25 16:39:11.345337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 705/1432] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:11 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:11.417509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:11.425977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:11.427487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:11.428640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6fccf94f-63a3-4d55-8e6b-3afa3abc31a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.179779 May 25 16:39:11.429194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 728/1433] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:11 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:12.270225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6d5a3727-eff2-44fb-a6d1-d4e67657465c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:12.270797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6d5a3727-eff2-44fb-a6d1-d4e67657465c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:12.271848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-266cc555-e469-4de6-9d0f-e3e462914272 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:12.272626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-266cc555-e469-4de6-9d0f-e3e462914272 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:12.277713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d5a3727-eff2-44fb-a6d1-d4e67657465c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:12.278166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d5a3727-eff2-44fb-a6d1-d4e67657465c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:12.278926 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d5a3727-eff2-44fb-a6d1-d4e67657465c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:12.279392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d5a3727-eff2-44fb-a6d1-d4e67657465c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:12.281234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-266cc555-e469-4de6-9d0f-e3e462914272 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:12.281746 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-266cc555-e469-4de6-9d0f-e3e462914272 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:12.282592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-266cc555-e469-4de6-9d0f-e3e462914272 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:12.283097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-266cc555-e469-4de6-9d0f-e3e462914272 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:12.479116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6d5a3727-eff2-44fb-a6d1-d4e67657465c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:12.482917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-266cc555-e469-4de6-9d0f-e3e462914272 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:12.487446 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6d5a3727-eff2-44fb-a6d1-d4e67657465c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.221423 May 25 16:39:12.488014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 706/1434] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:12 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:12.491146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-266cc555-e469-4de6-9d0f-e3e462914272 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.223409 May 25 16:39:12.501635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 729/1435] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:12 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:39:12.501809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-947d4eed-166b-41ad-8437-153136492ffa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:12.501809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-947d4eed-166b-41ad-8437-153136492ffa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:12.502015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:12.502015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:12.502220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-947d4eed-166b-41ad-8437-153136492ffa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:12.502220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-947d4eed-166b-41ad-8437-153136492ffa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:12.502220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-947d4eed-166b-41ad-8437-153136492ffa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:12.502220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-947d4eed-166b-41ad-8437-153136492ffa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:12.502758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:12.503239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:12.503998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:12.504439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:12.570447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-947d4eed-166b-41ad-8437-153136492ffa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:12.578875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-947d4eed-166b-41ad-8437-153136492ffa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089886 May 25 16:39:12.579445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 707/1436] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:12 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:12.656186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:12.665021 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:12.666522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:12.667723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c351ca53-8e05-4aee-8075-980e32c983d7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.175058 May 25 16:39:12.668290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 730/1437] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:12 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:13.508601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8fc8498b-0d19-41b0-b516-db3125aacd32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:13.509269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8fc8498b-0d19-41b0-b516-db3125aacd32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:13.511096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-74a4947e-9211-4617-8126-27f640e0abd8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:13.511768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-74a4947e-9211-4617-8126-27f640e0abd8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:13.516376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8fc8498b-0d19-41b0-b516-db3125aacd32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:13.516855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8fc8498b-0d19-41b0-b516-db3125aacd32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:13.517664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8fc8498b-0d19-41b0-b516-db3125aacd32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:13.518120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8fc8498b-0d19-41b0-b516-db3125aacd32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:13.518762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74a4947e-9211-4617-8126-27f640e0abd8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:13.519263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74a4947e-9211-4617-8126-27f640e0abd8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:13.520042 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74a4947e-9211-4617-8126-27f640e0abd8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:13.520503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-74a4947e-9211-4617-8126-27f640e0abd8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:13.712939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8fc8498b-0d19-41b0-b516-db3125aacd32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:13.717383 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-74a4947e-9211-4617-8126-27f640e0abd8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:13.721508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8fc8498b-0d19-41b0-b516-db3125aacd32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.218913 May 25 16:39:13.722078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 708/1438] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:13 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:13.726117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-74a4947e-9211-4617-8126-27f640e0abd8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.219367 May 25 16:39:13.726566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-579ef721-a58d-4f8f-a8b5-f7632a32cc10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:13.731023 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 731/1439] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:13 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:13.731279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-579ef721-a58d-4f8f-a8b5-f7632a32cc10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:13.731689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:13.732354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:13.733339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-579ef721-a58d-4f8f-a8b5-f7632a32cc10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:13.733807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-579ef721-a58d-4f8f-a8b5-f7632a32cc10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:13.734566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-579ef721-a58d-4f8f-a8b5-f7632a32cc10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:13.734992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-579ef721-a58d-4f8f-a8b5-f7632a32cc10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:13.738730 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:13.739344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:13.740592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:13.741218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:13.809607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-579ef721-a58d-4f8f-a8b5-f7632a32cc10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:13.818150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-579ef721-a58d-4f8f-a8b5-f7632a32cc10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.095152 May 25 16:39:13.818736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 709/1440] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:13 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:13.895510 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:13.905299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:13.906796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:13.907998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f4bb73d7-d555-4b75-98b1-d326421bf3f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.179911 May 25 16:39:13.908558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 732/1441] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:13 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:14.742920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4668b9e7-647d-455f-aa48-f78e6ca4ea53 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:14.743934 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4668b9e7-647d-455f-aa48-f78e6ca4ea53 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:14.747634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d646f9a3-2944-4609-a1ae-edd862968e10 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:14.748772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d646f9a3-2944-4609-a1ae-edd862968e10 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:14.756023 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4668b9e7-647d-455f-aa48-f78e6ca4ea53 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:14.756764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4668b9e7-647d-455f-aa48-f78e6ca4ea53 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:14.758059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4668b9e7-647d-455f-aa48-f78e6ca4ea53 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:14.758830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4668b9e7-647d-455f-aa48-f78e6ca4ea53 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:14.759679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d646f9a3-2944-4609-a1ae-edd862968e10 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:14.760429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d646f9a3-2944-4609-a1ae-edd862968e10 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:14.761666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d646f9a3-2944-4609-a1ae-edd862968e10 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:14.762386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d646f9a3-2944-4609-a1ae-edd862968e10 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:14.966781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d646f9a3-2944-4609-a1ae-edd862968e10 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:14.970292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4668b9e7-647d-455f-aa48-f78e6ca4ea53 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:14.975190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d646f9a3-2944-4609-a1ae-edd862968e10 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.234124 May 25 16:39:14.975767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 733/1442] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:14 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:14.980357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4668b9e7-647d-455f-aa48-f78e6ca4ea53 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.244026 May 25 16:39:14.981020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 710/1443] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:14 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:14.982017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-47115469-69ee-4c43-ab00-0f9c776cd00f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:14.983219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-47115469-69ee-4c43-ab00-0f9c776cd00f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:14.986790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:14.987058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:14.990091 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-47115469-69ee-4c43-ab00-0f9c776cd00f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:14.990516 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-47115469-69ee-4c43-ab00-0f9c776cd00f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:14.991332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-47115469-69ee-4c43-ab00-0f9c776cd00f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:14.991769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-47115469-69ee-4c43-ab00-0f9c776cd00f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:14.993786 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:14.994181 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:14.994945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:14.995421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:15.078762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-47115469-69ee-4c43-ab00-0f9c776cd00f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:15.087335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-47115469-69ee-4c43-ab00-0f9c776cd00f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.110277 May 25 16:39:15.087783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 734/1444] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:14 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:15.148063 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:15.150258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:15.161123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:15.163608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:15.165325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f16eb303-07f9-495d-9385-37b4f5ce0572 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.183124 May 25 16:39:15.165881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 711/1445] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:14 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:15.996247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a33c38a7-d085-410d-be97-d72ca34da476 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:15.996890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a33c38a7-d085-410d-be97-d72ca34da476 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:16.000092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a88e24f8-31ef-4048-8b04-ac80a0d9c0a8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:16.000545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:16.000776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a88e24f8-31ef-4048-8b04-ac80a0d9c0a8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:16.006609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a33c38a7-d085-410d-be97-d72ca34da476 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:16.007058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a33c38a7-d085-410d-be97-d72ca34da476 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:16.007864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a33c38a7-d085-410d-be97-d72ca34da476 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:16.008299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a33c38a7-d085-410d-be97-d72ca34da476 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:16.011180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a88e24f8-31ef-4048-8b04-ac80a0d9c0a8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:16.011641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a88e24f8-31ef-4048-8b04-ac80a0d9c0a8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:16.012366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a88e24f8-31ef-4048-8b04-ac80a0d9c0a8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:16.012989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a88e24f8-31ef-4048-8b04-ac80a0d9c0a8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:16.214249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a33c38a7-d085-410d-be97-d72ca34da476 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:16.217727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a88e24f8-31ef-4048-8b04-ac80a0d9c0a8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:16.223208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a33c38a7-d085-410d-be97-d72ca34da476 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.232429 May 25 16:39:16.223582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 735/1446] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:15 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:16.226583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a88e24f8-31ef-4048-8b04-ac80a0d9c0a8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.232255 May 25 16:39:16.227329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 712/1447] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:15 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:16.228259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7dfa00a4-7cce-4989-907e-9b53b0e1b1d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:16.228919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7dfa00a4-7cce-4989-907e-9b53b0e1b1d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:16.235459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7dfa00a4-7cce-4989-907e-9b53b0e1b1d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:16.236147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:16.236831 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7dfa00a4-7cce-4989-907e-9b53b0e1b1d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:16.236971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:16.237721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7dfa00a4-7cce-4989-907e-9b53b0e1b1d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:16.238188 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7dfa00a4-7cce-4989-907e-9b53b0e1b1d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:16.242998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:16.243560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:16.244337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:16.244766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:16.319734 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7dfa00a4-7cce-4989-907e-9b53b0e1b1d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:16.327991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7dfa00a4-7cce-4989-907e-9b53b0e1b1d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.103326 May 25 16:39:16.328584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 736/1448] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:16 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:16.408921 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:16.418477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:16.420008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:16.421157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-17a3577d-c7bc-44a1-a48d-d41fc2f5b7e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.188570 May 25 16:39:16.421714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 713/1449] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:16 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:17.244100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-920b3a7b-9263-4c41-9657-ab7bf626a405 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:17.245048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-920b3a7b-9263-4c41-9657-ab7bf626a405 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:17.247606 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4f851a4c-e71b-4f62-a717-ac6db60e31d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:17.248342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4f851a4c-e71b-4f62-a717-ac6db60e31d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:17.256105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-920b3a7b-9263-4c41-9657-ab7bf626a405 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:17.256808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-920b3a7b-9263-4c41-9657-ab7bf626a405 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:17.258049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-920b3a7b-9263-4c41-9657-ab7bf626a405 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:17.258514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-920b3a7b-9263-4c41-9657-ab7bf626a405 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:17.258657 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f851a4c-e71b-4f62-a717-ac6db60e31d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:17.258967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f851a4c-e71b-4f62-a717-ac6db60e31d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:17.259748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f851a4c-e71b-4f62-a717-ac6db60e31d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:17.260182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f851a4c-e71b-4f62-a717-ac6db60e31d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:17.456600 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-920b3a7b-9263-4c41-9657-ab7bf626a405 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:17.459795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4f851a4c-e71b-4f62-a717-ac6db60e31d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:17.466053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-920b3a7b-9263-4c41-9657-ab7bf626a405 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.228398 May 25 16:39:17.466655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 737/1450] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:17 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:17.471419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-21b70d95-1510-4b11-b2e5-7504f561324b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:17.472019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4f851a4c-e71b-4f62-a717-ac6db60e31d9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.230870 May 25 16:39:17.472172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-21b70d95-1510-4b11-b2e5-7504f561324b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:17.472767 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 714/1451] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:17 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:17.481905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21b70d95-1510-4b11-b2e5-7504f561324b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:17.483070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21b70d95-1510-4b11-b2e5-7504f561324b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:17.483960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21b70d95-1510-4b11-b2e5-7504f561324b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:17.484463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21b70d95-1510-4b11-b2e5-7504f561324b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:17.488603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:17.489184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:17.495527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:17.495984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:17.498403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:17.498403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:17.502009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:17.573145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-21b70d95-1510-4b11-b2e5-7504f561324b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:17.581229 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-21b70d95-1510-4b11-b2e5-7504f561324b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.113555 May 25 16:39:17.581993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 738/1452] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:17 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:17.640218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:17.648925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:17.650426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:17.651591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3d3ce49f-a77c-4161-8826-9306c18cc649 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.168087 May 25 16:39:17.652149 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 715/1453] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:17 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:18.488613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-04d36a4f-c0a7-4739-afb9-300f2485492f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:18.489278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-04d36a4f-c0a7-4739-afb9-300f2485492f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:18.490775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-036f9864-c2cf-442d-bde6-4d5895ab14ea tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:18.491380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-036f9864-c2cf-442d-bde6-4d5895ab14ea tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:18.496946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04d36a4f-c0a7-4739-afb9-300f2485492f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:18.497403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04d36a4f-c0a7-4739-afb9-300f2485492f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:18.498173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04d36a4f-c0a7-4739-afb9-300f2485492f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:18.498335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-036f9864-c2cf-442d-bde6-4d5895ab14ea tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:18.498610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-04d36a4f-c0a7-4739-afb9-300f2485492f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:18.498784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-036f9864-c2cf-442d-bde6-4d5895ab14ea tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:18.499568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-036f9864-c2cf-442d-bde6-4d5895ab14ea tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:18.499985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-036f9864-c2cf-442d-bde6-4d5895ab14ea tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:18.693256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-036f9864-c2cf-442d-bde6-4d5895ab14ea tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:18.697378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-04d36a4f-c0a7-4739-afb9-300f2485492f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:18.701974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-036f9864-c2cf-442d-bde6-4d5895ab14ea tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.215647 May 25 16:39:18.702552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 716/1454] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:18 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:18.705574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-04d36a4f-c0a7-4739-afb9-300f2485492f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222158 May 25 16:39:18.711626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 739/1455] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:18 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:18.711801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d1e424b9-2821-446e-afbd-f6d231385692 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:18.711801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d1e424b9-2821-446e-afbd-f6d231385692 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:18.715420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:18.716310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:18.716460 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d1e424b9-2821-446e-afbd-f6d231385692 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:18.717115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d1e424b9-2821-446e-afbd-f6d231385692 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:18.718399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d1e424b9-2821-446e-afbd-f6d231385692 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:18.719077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d1e424b9-2821-446e-afbd-f6d231385692 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:18.724452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:18.724907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:18.725646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:18.726564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:18.807717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d1e424b9-2821-446e-afbd-f6d231385692 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:18.816104 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d1e424b9-2821-446e-afbd-f6d231385692 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.111774 May 25 16:39:18.816865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 717/1456] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:18 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:18.879352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:18.888488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:18.889987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:18.891186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8d957f02-184c-43ca-abbb-39795a9bbd4c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.184120 May 25 16:39:18.891759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 740/1457] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:19.723121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bc6918da-f92f-4377-a531-ab7f7ca3df25 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:19.724205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bc6918da-f92f-4377-a531-ab7f7ca3df25 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:19.727101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2813e3e6-a75b-47de-b63a-df9e9c5f3d16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:19.728339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2813e3e6-a75b-47de-b63a-df9e9c5f3d16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:19.735373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc6918da-f92f-4377-a531-ab7f7ca3df25 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:19.736141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc6918da-f92f-4377-a531-ab7f7ca3df25 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:19.737405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc6918da-f92f-4377-a531-ab7f7ca3df25 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:19.738132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bc6918da-f92f-4377-a531-ab7f7ca3df25 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:19.740285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2813e3e6-a75b-47de-b63a-df9e9c5f3d16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:19.741031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2813e3e6-a75b-47de-b63a-df9e9c5f3d16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:19.742171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2813e3e6-a75b-47de-b63a-df9e9c5f3d16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:19.742606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2813e3e6-a75b-47de-b63a-df9e9c5f3d16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:19.932135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bc6918da-f92f-4377-a531-ab7f7ca3df25 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:19.935184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2813e3e6-a75b-47de-b63a-df9e9c5f3d16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:19.940870 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bc6918da-f92f-4377-a531-ab7f7ca3df25 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.224145 May 25 16:39:19.941490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 718/1458] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:19 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:19.943775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2813e3e6-a75b-47de-b63a-df9e9c5f3d16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.223474 May 25 16:39:19.944308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 741/1459] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:19 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:39:19.948041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-86e0b13f-3933-4dba-a4d0-17a48ad288cf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:19.948869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-86e0b13f-3933-4dba-a4d0-17a48ad288cf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:19.949482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:19.950151 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:19.955774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86e0b13f-3933-4dba-a4d0-17a48ad288cf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:19.956225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86e0b13f-3933-4dba-a4d0-17a48ad288cf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:19.956978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86e0b13f-3933-4dba-a4d0-17a48ad288cf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:19.957253 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:19.957444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86e0b13f-3933-4dba-a4d0-17a48ad288cf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:19.957676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:19.958436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:19.958862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:20.039333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-86e0b13f-3933-4dba-a4d0-17a48ad288cf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:20.047971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-86e0b13f-3933-4dba-a4d0-17a48ad288cf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.105538 May 25 16:39:20.048543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 719/1460] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:19 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:20.105854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:20.115822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:20.117299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:20.118468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-24b54d68-c023-48bd-813a-da0d0e3854ec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.173002 May 25 16:39:20.118999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 742/1461] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:19 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:20.962571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-058c4367-4ab7-4765-ac99-fda85736618e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:20.963566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-058c4367-4ab7-4765-ac99-fda85736618e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:20.964410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-29a3adf4-e08c-4266-aaf9-24841a06c1cf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:20.965359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-29a3adf4-e08c-4266-aaf9-24841a06c1cf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:20.974052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-058c4367-4ab7-4765-ac99-fda85736618e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:20.974657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29a3adf4-e08c-4266-aaf9-24841a06c1cf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:20.974862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-058c4367-4ab7-4765-ac99-fda85736618e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:20.975389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29a3adf4-e08c-4266-aaf9-24841a06c1cf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:20.975930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-058c4367-4ab7-4765-ac99-fda85736618e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:20.976541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29a3adf4-e08c-4266-aaf9-24841a06c1cf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:20.976701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-058c4367-4ab7-4765-ac99-fda85736618e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:20.977207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29a3adf4-e08c-4266-aaf9-24841a06c1cf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:21.173156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-29a3adf4-e08c-4266-aaf9-24841a06c1cf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:21.176576 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-058c4367-4ab7-4765-ac99-fda85736618e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:21.181588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-29a3adf4-e08c-4266-aaf9-24841a06c1cf tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.223238 May 25 16:39:21.182275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 743/1462] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:20 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:21.185961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-058c4367-4ab7-4765-ac99-fda85736618e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.229844 May 25 16:39:21.186791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 720/1463] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:20 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:21.186919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-59965efd-d48e-4c80-803a-943705a4b788 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:21.187654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-59965efd-d48e-4c80-803a-943705a4b788 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:21.192535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:21.193470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:21.194120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59965efd-d48e-4c80-803a-943705a4b788 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:21.194636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59965efd-d48e-4c80-803a-943705a4b788 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:21.195521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59965efd-d48e-4c80-803a-943705a4b788 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:21.196024 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59965efd-d48e-4c80-803a-943705a4b788 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:21.200390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:21.200919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:21.201734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:21.202211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:21.264660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-59965efd-d48e-4c80-803a-943705a4b788 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:21.273225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-59965efd-d48e-4c80-803a-943705a4b788 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089830 May 25 16:39:21.274028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 744/1464] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:21 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:21.353941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:21.367496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:21.368997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:21.370166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-afc8310e-1ff4-466b-af8e-6c225bf811b0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.182086 May 25 16:39:21.370722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 721/1465] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:21 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:22.201421 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-43c77a8c-7072-47b8-8aa5-3c5537882131 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:22.202045 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-43c77a8c-7072-47b8-8aa5-3c5537882131 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:22.205272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1f7bd83e-bb33-4724-97a2-4236a74f4791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:22.206383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1f7bd83e-bb33-4724-97a2-4236a74f4791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:22.209234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-43c77a8c-7072-47b8-8aa5-3c5537882131 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:22.209689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-43c77a8c-7072-47b8-8aa5-3c5537882131 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:22.210452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-43c77a8c-7072-47b8-8aa5-3c5537882131 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:22.210885 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-43c77a8c-7072-47b8-8aa5-3c5537882131 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:22.213088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1f7bd83e-bb33-4724-97a2-4236a74f4791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:22.213724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1f7bd83e-bb33-4724-97a2-4236a74f4791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:22.214474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1f7bd83e-bb33-4724-97a2-4236a74f4791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:22.214923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1f7bd83e-bb33-4724-97a2-4236a74f4791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:22.407905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-43c77a8c-7072-47b8-8aa5-3c5537882131 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:22.411389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1f7bd83e-bb33-4724-97a2-4236a74f4791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:22.416552 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-43c77a8c-7072-47b8-8aa5-3c5537882131 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220568 May 25 16:39:22.417202 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 745/1466] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:22 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:22.421689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f86fbc68-30e7-4ac0-bc47-f0e1ff801824 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:22.422295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f86fbc68-30e7-4ac0-bc47-f0e1ff801824 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:22.423858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1f7bd83e-bb33-4724-97a2-4236a74f4791 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.223363 May 25 16:39:22.431700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 722/1467] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:22 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:22.431700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:22.431700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:22.435174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f86fbc68-30e7-4ac0-bc47-f0e1ff801824 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:22.435739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f86fbc68-30e7-4ac0-bc47-f0e1ff801824 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:22.436070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:22.436551 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:22.436693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f86fbc68-30e7-4ac0-bc47-f0e1ff801824 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:22.437074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f86fbc68-30e7-4ac0-bc47-f0e1ff801824 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:22.437342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:22.437772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:22.526015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f86fbc68-30e7-4ac0-bc47-f0e1ff801824 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:22.534641 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f86fbc68-30e7-4ac0-bc47-f0e1ff801824 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.116549 May 25 16:39:22.535255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 746/1468] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:22 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:22.602632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:22.616208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:22.618499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:22.620307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dd853d0b-71ed-445f-8eb8-7d5d52b3db5c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.194538 May 25 16:39:22.621038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 723/1469] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:22 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:23.438225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ec0c3717-f40e-4f64-ad9b-bf87993a0aa4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:23.439201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ec0c3717-f40e-4f64-ad9b-bf87993a0aa4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:23.444381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-755b804c-1974-4a1c-9208-b5f4391cacdd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:23.444994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-755b804c-1974-4a1c-9208-b5f4391cacdd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:23.446444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec0c3717-f40e-4f64-ad9b-bf87993a0aa4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:23.446950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec0c3717-f40e-4f64-ad9b-bf87993a0aa4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:23.447777 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec0c3717-f40e-4f64-ad9b-bf87993a0aa4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:23.448207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec0c3717-f40e-4f64-ad9b-bf87993a0aa4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:23.451637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-755b804c-1974-4a1c-9208-b5f4391cacdd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:23.452098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-755b804c-1974-4a1c-9208-b5f4391cacdd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:23.452877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-755b804c-1974-4a1c-9208-b5f4391cacdd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:23.453325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-755b804c-1974-4a1c-9208-b5f4391cacdd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:23.638728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ec0c3717-f40e-4f64-ad9b-bf87993a0aa4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:23.645111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-755b804c-1974-4a1c-9208-b5f4391cacdd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:23.648503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ec0c3717-f40e-4f64-ad9b-bf87993a0aa4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217115 May 25 16:39:23.649095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 747/1470] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:23 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:23.653696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-755b804c-1974-4a1c-9208-b5f4391cacdd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.213057 May 25 16:39:23.653867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e6fed32b-93fd-4026-baab-bb8f81fe79ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:23.663652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 724/1471] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:23 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:23.663865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e6fed32b-93fd-4026-baab-bb8f81fe79ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:23.663986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:23.663986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:23.664181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6fed32b-93fd-4026-baab-bb8f81fe79ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:23.664181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6fed32b-93fd-4026-baab-bb8f81fe79ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:23.664181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6fed32b-93fd-4026-baab-bb8f81fe79ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:23.664181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6fed32b-93fd-4026-baab-bb8f81fe79ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:23.666312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:23.666760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:23.667618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:23.668047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:23.753722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e6fed32b-93fd-4026-baab-bb8f81fe79ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:23.762928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e6fed32b-93fd-4026-baab-bb8f81fe79ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.112864 May 25 16:39:23.763711 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 748/1472] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:23 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:23.821798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:23.830558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:23.832120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:23.833279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-959e17fc-2cb3-45ee-9479-93ffa3683720 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.178103 May 25 16:39:23.833838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 725/1473] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:23 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:24.668850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-62a5b93f-b22a-479a-92e5-ea53608bebbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:24.669685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-62a5b93f-b22a-479a-92e5-ea53608bebbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:24.671925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e0eb0165-ad2b-4b6b-b1bb-2222e3ce5597 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:24.672560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e0eb0165-ad2b-4b6b-b1bb-2222e3ce5597 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:24.677874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62a5b93f-b22a-479a-92e5-ea53608bebbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:24.678374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62a5b93f-b22a-479a-92e5-ea53608bebbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:24.679218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62a5b93f-b22a-479a-92e5-ea53608bebbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:24.679357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0eb0165-ad2b-4b6b-b1bb-2222e3ce5597 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:24.679750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62a5b93f-b22a-479a-92e5-ea53608bebbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:24.679897 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0eb0165-ad2b-4b6b-b1bb-2222e3ce5597 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:24.680558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0eb0165-ad2b-4b6b-b1bb-2222e3ce5597 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:24.680986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0eb0165-ad2b-4b6b-b1bb-2222e3ce5597 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:24.876982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e0eb0165-ad2b-4b6b-b1bb-2222e3ce5597 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:24.881065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-62a5b93f-b22a-479a-92e5-ea53608bebbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:24.885928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e0eb0165-ad2b-4b6b-b1bb-2222e3ce5597 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.218180 May 25 16:39:24.886636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 726/1474] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:24 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:24.889785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-62a5b93f-b22a-479a-92e5-ea53608bebbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.225758 May 25 16:39:24.891279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 749/1475] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:24 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:24.891816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1cacff3b-e175-4683-883c-ebbc5e0b0afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:24.892538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1cacff3b-e175-4683-883c-ebbc5e0b0afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:24.897577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:24.898188 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:24.899144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1cacff3b-e175-4683-883c-ebbc5e0b0afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:24.899688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1cacff3b-e175-4683-883c-ebbc5e0b0afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:24.900613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1cacff3b-e175-4683-883c-ebbc5e0b0afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:24.901065 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1cacff3b-e175-4683-883c-ebbc5e0b0afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:24.904453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:24.904899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:24.905664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:24.906097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:24.987363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1cacff3b-e175-4683-883c-ebbc5e0b0afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:24.996221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1cacff3b-e175-4683-883c-ebbc5e0b0afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.108584 May 25 16:39:24.996759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 727/1476] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:24 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:25.056700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:25.065669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:25.067630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:25.068829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-78a396eb-9d3b-47a1-ad9b-ef9a2f62aefa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.176033 May 25 16:39:25.069412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 750/1477] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:24 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:25.905454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c2aa48f3-bfdd-40a3-aaa2-358a6b0bfd78 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:25.906235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c2aa48f3-bfdd-40a3-aaa2-358a6b0bfd78 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:25.908154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-937bef9d-99a7-48bd-a39b-9050d4cf1714 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:25.908943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-937bef9d-99a7-48bd-a39b-9050d4cf1714 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:25.917898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c2aa48f3-bfdd-40a3-aaa2-358a6b0bfd78 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:25.918662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c2aa48f3-bfdd-40a3-aaa2-358a6b0bfd78 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:25.919480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-937bef9d-99a7-48bd-a39b-9050d4cf1714 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:25.920097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c2aa48f3-bfdd-40a3-aaa2-358a6b0bfd78 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:25.920272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-937bef9d-99a7-48bd-a39b-9050d4cf1714 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:25.920829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c2aa48f3-bfdd-40a3-aaa2-358a6b0bfd78 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:25.921178 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-937bef9d-99a7-48bd-a39b-9050d4cf1714 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:25.921799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-937bef9d-99a7-48bd-a39b-9050d4cf1714 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:25.938068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:26.127601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c2aa48f3-bfdd-40a3-aaa2-358a6b0bfd78 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:26.130813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-937bef9d-99a7-48bd-a39b-9050d4cf1714 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:26.135909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c2aa48f3-bfdd-40a3-aaa2-358a6b0bfd78 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.235380 May 25 16:39:26.136618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 728/1478] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:25 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:26.139967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-937bef9d-99a7-48bd-a39b-9050d4cf1714 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.237100 May 25 16:39:26.140820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 751/1479] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:25 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:26.145321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:26.145991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:26.149693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7e4b0a24-444e-4e1e-84b9-7bbae73ed5f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:26.150345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7e4b0a24-444e-4e1e-84b9-7bbae73ed5f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:26.152217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:26.152662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:26.153452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:26.153877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:26.156478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e4b0a24-444e-4e1e-84b9-7bbae73ed5f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:26.156877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e4b0a24-444e-4e1e-84b9-7bbae73ed5f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:26.157634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e4b0a24-444e-4e1e-84b9-7bbae73ed5f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:26.159010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e4b0a24-444e-4e1e-84b9-7bbae73ed5f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:26.255184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7e4b0a24-444e-4e1e-84b9-7bbae73ed5f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:26.263711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7e4b0a24-444e-4e1e-84b9-7bbae73ed5f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.126121 May 25 16:39:26.264326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 729/1480] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:26.324198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:26.337308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:26.339415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:26.341231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cec3c564-90e8-4ed9-b27b-f75882dfef54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.199243 May 25 16:39:26.341973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 752/1481] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:27.158288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b0fa8b4f-a8cc-4316-9dd8-0a02c146b2e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:27.159133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b0fa8b4f-a8cc-4316-9dd8-0a02c146b2e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:27.160119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4af6ac7e-21f7-4c16-b398-39072a08f8ff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:27.161037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4af6ac7e-21f7-4c16-b398-39072a08f8ff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:27.167398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0fa8b4f-a8cc-4316-9dd8-0a02c146b2e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:27.167846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0fa8b4f-a8cc-4316-9dd8-0a02c146b2e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:27.168257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4af6ac7e-21f7-4c16-b398-39072a08f8ff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:27.168596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0fa8b4f-a8cc-4316-9dd8-0a02c146b2e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:27.168714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4af6ac7e-21f7-4c16-b398-39072a08f8ff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:27.169025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0fa8b4f-a8cc-4316-9dd8-0a02c146b2e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:27.169437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4af6ac7e-21f7-4c16-b398-39072a08f8ff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:27.169860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4af6ac7e-21f7-4c16-b398-39072a08f8ff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:27.362485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b0fa8b4f-a8cc-4316-9dd8-0a02c146b2e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:27.365029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4af6ac7e-21f7-4c16-b398-39072a08f8ff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:27.371882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b0fa8b4f-a8cc-4316-9dd8-0a02c146b2e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.220569 May 25 16:39:27.372440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 730/1482] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:27 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:27.380887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4af6ac7e-21f7-4c16-b398-39072a08f8ff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.219651 May 25 16:39:27.380887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 753/1483] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:27 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:27.381125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-00a80966-9ea7-40ad-babc-71daab23ded0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:27.381229 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:27.381332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-00a80966-9ea7-40ad-babc-71daab23ded0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:27.381424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:27.385318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00a80966-9ea7-40ad-babc-71daab23ded0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:27.385801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00a80966-9ea7-40ad-babc-71daab23ded0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:27.386540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:27.386712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00a80966-9ea7-40ad-babc-71daab23ded0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:27.386982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:27.387130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00a80966-9ea7-40ad-babc-71daab23ded0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:27.387760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:27.388187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:27.456912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-00a80966-9ea7-40ad-babc-71daab23ded0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:27.465437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-00a80966-9ea7-40ad-babc-71daab23ded0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091780 May 25 16:39:27.466376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 731/1484] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:27 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:27.543947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:27.553143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:27.554644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:27.555882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-62e03722-3295-4b61-9416-9d2dce608ef5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.180867 May 25 16:39:27.556478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 754/1485] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:28.389925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bf16998e-751d-45b8-856a-c83774fb4a61 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:28.390472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-65d40383-8244-44b3-a7e0-205b05ea444f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:28.390849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bf16998e-751d-45b8-856a-c83774fb4a61 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:28.390957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-65d40383-8244-44b3-a7e0-205b05ea444f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:28.397214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-65d40383-8244-44b3-a7e0-205b05ea444f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:28.397552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf16998e-751d-45b8-856a-c83774fb4a61 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:28.397670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-65d40383-8244-44b3-a7e0-205b05ea444f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:28.398014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf16998e-751d-45b8-856a-c83774fb4a61 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:28.398493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-65d40383-8244-44b3-a7e0-205b05ea444f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:28.398738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf16998e-751d-45b8-856a-c83774fb4a61 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:28.398855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-65d40383-8244-44b3-a7e0-205b05ea444f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:28.399176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf16998e-751d-45b8-856a-c83774fb4a61 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:28.403957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:28.588906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-65d40383-8244-44b3-a7e0-205b05ea444f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:28.598185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-65d40383-8244-44b3-a7e0-205b05ea444f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.212316 May 25 16:39:28.608565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 755/1486] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:28 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:39:28.608565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cd55fef2-9176-4f60-99be-c3f1b4f2d6cb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:28.608565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cd55fef2-9176-4f60-99be-c3f1b4f2d6cb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:28.609087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bf16998e-751d-45b8-856a-c83774fb4a61 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:28.610426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd55fef2-9176-4f60-99be-c3f1b4f2d6cb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:28.610882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd55fef2-9176-4f60-99be-c3f1b4f2d6cb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:28.611663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd55fef2-9176-4f60-99be-c3f1b4f2d6cb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:28.612114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd55fef2-9176-4f60-99be-c3f1b4f2d6cb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:28.613890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bf16998e-751d-45b8-856a-c83774fb4a61 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.228180 May 25 16:39:28.614527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 732/1487] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:28 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:28.619082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:28.619767 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:28.625812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:28.626262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:28.627078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:28.627531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:28.675500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cd55fef2-9176-4f60-99be-c3f1b4f2d6cb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:28.683617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cd55fef2-9176-4f60-99be-c3f1b4f2d6cb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083635 May 25 16:39:28.684194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 756/1488] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:28.782061 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:28.796661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:28.799049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:28.800900 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bf143845-1301-4268-ad3e-beeff261a47a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.185068 May 25 16:39:28.801666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 733/1489] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:29.619638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-59c60cd0-86a6-42ed-a26d-a1a39fae5505 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:29.620530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-59c60cd0-86a6-42ed-a26d-a1a39fae5505 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:29.630222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59c60cd0-86a6-42ed-a26d-a1a39fae5505 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:29.630777 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59c60cd0-86a6-42ed-a26d-a1a39fae5505 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:29.631937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59c60cd0-86a6-42ed-a26d-a1a39fae5505 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:29.632598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59c60cd0-86a6-42ed-a26d-a1a39fae5505 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:29.634058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-70c1a83c-8342-48d7-bd51-5bf959391ff2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:29.634808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-70c1a83c-8342-48d7-bd51-5bf959391ff2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:29.643629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70c1a83c-8342-48d7-bd51-5bf959391ff2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:29.644068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70c1a83c-8342-48d7-bd51-5bf959391ff2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:29.644820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70c1a83c-8342-48d7-bd51-5bf959391ff2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:29.645251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70c1a83c-8342-48d7-bd51-5bf959391ff2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:29.828089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-59c60cd0-86a6-42ed-a26d-a1a39fae5505 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:29.831098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-70c1a83c-8342-48d7-bd51-5bf959391ff2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:29.836544 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-59c60cd0-86a6-42ed-a26d-a1a39fae5505 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222504 May 25 16:39:29.837124 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 757/1490] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:29 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:29.839357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-70c1a83c-8342-48d7-bd51-5bf959391ff2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.210005 May 25 16:39:29.840415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 734/1491] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:29 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:29.841549 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0039a3c1-2932-4b9d-ab16-68a7de5c88ff tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:29.842160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0039a3c1-2932-4b9d-ab16-68a7de5c88ff tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:29.851013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:29.851645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:29.852570 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0039a3c1-2932-4b9d-ab16-68a7de5c88ff tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:29.853055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0039a3c1-2932-4b9d-ab16-68a7de5c88ff tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:29.853779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0039a3c1-2932-4b9d-ab16-68a7de5c88ff tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:29.854235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0039a3c1-2932-4b9d-ab16-68a7de5c88ff tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:29.857639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:29.858091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:29.858813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:29.859270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:29.926973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0039a3c1-2932-4b9d-ab16-68a7de5c88ff tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:29.935105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0039a3c1-2932-4b9d-ab16-68a7de5c88ff tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.097049 May 25 16:39:29.935927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 758/1492] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:29 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:30.016029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:30.025158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:30.026662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:30.027859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-db4d11c9-4bf2-4657-bcfd-ce2ba1ce167a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.182078 May 25 16:39:30.028436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 735/1493] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:29 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:30.858107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a0eb9a43-27c4-4ed3-814e-dbf536b67b20 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:30.858923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a0eb9a43-27c4-4ed3-814e-dbf536b67b20 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:30.859357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8b826635-1880-4d4c-8e5e-ecc31a25b903 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:30.860131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8b826635-1880-4d4c-8e5e-ecc31a25b903 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:30.867016 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0eb9a43-27c4-4ed3-814e-dbf536b67b20 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:30.867572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0eb9a43-27c4-4ed3-814e-dbf536b67b20 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:30.867894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b826635-1880-4d4c-8e5e-ecc31a25b903 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:30.868477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b826635-1880-4d4c-8e5e-ecc31a25b903 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:30.868608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0eb9a43-27c4-4ed3-814e-dbf536b67b20 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:30.868980 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0eb9a43-27c4-4ed3-814e-dbf536b67b20 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:30.869455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b826635-1880-4d4c-8e5e-ecc31a25b903 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:30.870018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b826635-1880-4d4c-8e5e-ecc31a25b903 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:31.302859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a0eb9a43-27c4-4ed3-814e-dbf536b67b20 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:31.308896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8b826635-1880-4d4c-8e5e-ecc31a25b903 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:31.316642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a0eb9a43-27c4-4ed3-814e-dbf536b67b20 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.462762 May 25 16:39:31.317433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 736/1494] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:30 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:31.321094 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8b826635-1880-4d4c-8e5e-ecc31a25b903 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.467438 May 25 16:39:31.325650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 759/1495] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:30 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:31.325795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5841c495-7be6-480d-93a6-aa973154f7ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:31.325795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5841c495-7be6-480d-93a6-aa973154f7ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:31.327577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:31.328192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:31.330084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5841c495-7be6-480d-93a6-aa973154f7ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:31.330559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5841c495-7be6-480d-93a6-aa973154f7ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:31.331419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5841c495-7be6-480d-93a6-aa973154f7ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:31.331817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5841c495-7be6-480d-93a6-aa973154f7ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:31.335790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:31.336243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:31.337128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:31.337621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:31.405267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5841c495-7be6-480d-93a6-aa973154f7ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:31.414024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5841c495-7be6-480d-93a6-aa973154f7ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.095547 May 25 16:39:31.415119 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 737/1496] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:31 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:31.490597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:31.500837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:31.502377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:31.503553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ea4559eb-fcc1-4137-9d94-f862e9c9e952 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.180855 May 25 16:39:31.504115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 760/1497] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:31 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:32.340686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-09d556e2-75f7-4f72-be41-232b225719f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:32.341455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-644e378b-972a-49db-9826-fbe411a08a75 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:32.342060 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-644e378b-972a-49db-9826-fbe411a08a75 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:32.342211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-09d556e2-75f7-4f72-be41-232b225719f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:32.349543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-644e378b-972a-49db-9826-fbe411a08a75 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:32.349996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-644e378b-972a-49db-9826-fbe411a08a75 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:32.350159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-09d556e2-75f7-4f72-be41-232b225719f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:32.350470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-09d556e2-75f7-4f72-be41-232b225719f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:32.350766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-644e378b-972a-49db-9826-fbe411a08a75 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:32.351246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-644e378b-972a-49db-9826-fbe411a08a75 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:32.351402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-09d556e2-75f7-4f72-be41-232b225719f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:32.351733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-09d556e2-75f7-4f72-be41-232b225719f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:32.551465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-09d556e2-75f7-4f72-be41-232b225719f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:32.555621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-644e378b-972a-49db-9826-fbe411a08a75 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:32.559592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-09d556e2-75f7-4f72-be41-232b225719f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.226162 May 25 16:39:32.560132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 738/1498] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:32 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:32.564497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-644e378b-972a-49db-9826-fbe411a08a75 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.229448 May 25 16:39:32.565089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fcda6c75-17f9-4a62-b282-8f9119bf83f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:32.565367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 761/1499] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:32 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:32.570014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:32.570611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:32.574139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fcda6c75-17f9-4a62-b282-8f9119bf83f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:32.577300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:32.577748 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:32.578514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:32.578978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:32.580458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fcda6c75-17f9-4a62-b282-8f9119bf83f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:32.581028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fcda6c75-17f9-4a62-b282-8f9119bf83f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:32.581799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fcda6c75-17f9-4a62-b282-8f9119bf83f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:32.582275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fcda6c75-17f9-4a62-b282-8f9119bf83f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:32.661793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fcda6c75-17f9-4a62-b282-8f9119bf83f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:32.670128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fcda6c75-17f9-4a62-b282-8f9119bf83f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.108814 May 25 16:39:32.670699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 739/1500] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:32 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:32.743652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:32.752812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:32.754344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:32.755566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5c12d546-f092-4010-88cb-fce825f8dd25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.189188 May 25 16:39:32.756131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 762/1501] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:32 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:33.580604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3207d0c4-adf6-4c88-b270-41f97cf23c1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:33.581207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3207d0c4-adf6-4c88-b270-41f97cf23c1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:33.583619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5cdcd9d5-3788-4c44-ab44-1b5b14103704 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:33.584298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5cdcd9d5-3788-4c44-ab44-1b5b14103704 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:33.588180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3207d0c4-adf6-4c88-b270-41f97cf23c1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:33.588631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3207d0c4-adf6-4c88-b270-41f97cf23c1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:33.589400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3207d0c4-adf6-4c88-b270-41f97cf23c1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:33.589856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3207d0c4-adf6-4c88-b270-41f97cf23c1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:33.591461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5cdcd9d5-3788-4c44-ab44-1b5b14103704 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:33.591976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5cdcd9d5-3788-4c44-ab44-1b5b14103704 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:33.592881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5cdcd9d5-3788-4c44-ab44-1b5b14103704 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:33.593341 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5cdcd9d5-3788-4c44-ab44-1b5b14103704 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:33.783053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3207d0c4-adf6-4c88-b270-41f97cf23c1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:33.787529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5cdcd9d5-3788-4c44-ab44-1b5b14103704 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:33.791426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3207d0c4-adf6-4c88-b270-41f97cf23c1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.217231 May 25 16:39:33.791991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 740/1502] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:33 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:33.796619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5cdcd9d5-3788-4c44-ab44-1b5b14103704 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217299 May 25 16:39:33.796813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7dde166a-163e-4f0b-bc35-3acbc6719c0d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:33.797398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 763/1503] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:33 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:33.797533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7dde166a-163e-4f0b-bc35-3acbc6719c0d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:33.803597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dde166a-163e-4f0b-bc35-3acbc6719c0d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:33.804046 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dde166a-163e-4f0b-bc35-3acbc6719c0d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:33.804801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dde166a-163e-4f0b-bc35-3acbc6719c0d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:33.805229 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dde166a-163e-4f0b-bc35-3acbc6719c0d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:33.806432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:33.807038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:33.813142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:33.813585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:33.814433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:33.814867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:33.879111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7dde166a-163e-4f0b-bc35-3acbc6719c0d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:33.887357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7dde166a-163e-4f0b-bc35-3acbc6719c0d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.094354 May 25 16:39:33.887924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 741/1504] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:33 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:33.966480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:33.975186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:33.976705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:33.977876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e6ebde54-d98e-4d37-a4d2-9174ee081c08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.175095 May 25 16:39:33.978436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 764/1505] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:33 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:34.813633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dfb6877a-a60d-43a1-b6e2-e7220b7e3244 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:34.814510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dfb6877a-a60d-43a1-b6e2-e7220b7e3244 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:34.816408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ac1855a5-9cda-45e2-8693-291921c4ba32 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:34.817054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ac1855a5-9cda-45e2-8693-291921c4ba32 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:34.821835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dfb6877a-a60d-43a1-b6e2-e7220b7e3244 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:34.822314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dfb6877a-a60d-43a1-b6e2-e7220b7e3244 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:34.823183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dfb6877a-a60d-43a1-b6e2-e7220b7e3244 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:34.823670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac1855a5-9cda-45e2-8693-291921c4ba32 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:34.823852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dfb6877a-a60d-43a1-b6e2-e7220b7e3244 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:34.824089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac1855a5-9cda-45e2-8693-291921c4ba32 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:34.824886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac1855a5-9cda-45e2-8693-291921c4ba32 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:34.825323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac1855a5-9cda-45e2-8693-291921c4ba32 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:35.035820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dfb6877a-a60d-43a1-b6e2-e7220b7e3244 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:35.038894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ac1855a5-9cda-45e2-8693-291921c4ba32 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:35.044987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dfb6877a-a60d-43a1-b6e2-e7220b7e3244 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.238383 May 25 16:39:35.045828 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 742/1506] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:34 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:35.051356 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-38476e00-61cb-4c43-8b4a-c3f7ab29ce06 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:35.052111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-38476e00-61cb-4c43-8b4a-c3f7ab29ce06 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:35.054823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ac1855a5-9cda-45e2-8693-291921c4ba32 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.243692 May 25 16:39:35.055488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 765/1507] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:34 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:35.059089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-38476e00-61cb-4c43-8b4a-c3f7ab29ce06 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:35.059730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-38476e00-61cb-4c43-8b4a-c3f7ab29ce06 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:35.060073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:35.060602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-38476e00-61cb-4c43-8b4a-c3f7ab29ce06 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:35.060933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:35.061052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-38476e00-61cb-4c43-8b4a-c3f7ab29ce06 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:35.067966 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:35.068306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:35.069474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:35.069474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:35.152503 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-38476e00-61cb-4c43-8b4a-c3f7ab29ce06 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:35.163110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-38476e00-61cb-4c43-8b4a-c3f7ab29ce06 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.116070 May 25 16:39:35.163933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 743/1508] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:35 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:35.228426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:35.237227 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:35.238727 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:35.239915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0bdba3e8-faf7-444a-b589-a520e59fe8d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.183486 May 25 16:39:35.240491 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 766/1509] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:35 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:36.066071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c8518bfa-1a2e-44a4-a9a7-ead32d0cb048 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:36.067234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c8518bfa-1a2e-44a4-a9a7-ead32d0cb048 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:36.076974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-865cf6d5-58e7-4d23-ac27-256ed27f0055 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:36.077987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-865cf6d5-58e7-4d23-ac27-256ed27f0055 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:36.079220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8518bfa-1a2e-44a4-a9a7-ead32d0cb048 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:36.079993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8518bfa-1a2e-44a4-a9a7-ead32d0cb048 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:36.081618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8518bfa-1a2e-44a4-a9a7-ead32d0cb048 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:36.082183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c8518bfa-1a2e-44a4-a9a7-ead32d0cb048 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:36.086404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-865cf6d5-58e7-4d23-ac27-256ed27f0055 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:36.086855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-865cf6d5-58e7-4d23-ac27-256ed27f0055 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:36.087639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-865cf6d5-58e7-4d23-ac27-256ed27f0055 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:36.088067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-865cf6d5-58e7-4d23-ac27-256ed27f0055 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:36.276589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c8518bfa-1a2e-44a4-a9a7-ead32d0cb048 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:36.280358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-865cf6d5-58e7-4d23-ac27-256ed27f0055 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:36.285939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c8518bfa-1a2e-44a4-a9a7-ead32d0cb048 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.226497 May 25 16:39:36.286520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 744/1510] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:36 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:36.291267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ef7c3df1-5968-4ca1-9e09-fa4cfd98cd52 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:36.291872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ef7c3df1-5968-4ca1-9e09-fa4cfd98cd52 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:36.293911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-865cf6d5-58e7-4d23-ac27-256ed27f0055 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.223791 May 25 16:39:36.294920 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 767/1511] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:36 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:36.299515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:36.300131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:36.303530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef7c3df1-5968-4ca1-9e09-fa4cfd98cd52 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:36.303998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef7c3df1-5968-4ca1-9e09-fa4cfd98cd52 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:36.304821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef7c3df1-5968-4ca1-9e09-fa4cfd98cd52 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:36.305267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef7c3df1-5968-4ca1-9e09-fa4cfd98cd52 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:36.305964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:36.306402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:36.307142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:36.307591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:36.309410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:36.396070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ef7c3df1-5968-4ca1-9e09-fa4cfd98cd52 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:36.404416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ef7c3df1-5968-4ca1-9e09-fa4cfd98cd52 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.116941 May 25 16:39:36.405411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 745/1512] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:36 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:36.465138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:36.474240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:36.477345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:36.478638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-939720f9-0922-4295-9e11-c3edc10406f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.182668 May 25 16:39:36.479243 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 768/1513] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:36 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:37.306596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7c298fe9-1ac4-4bae-a054-4a72b4daa68e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:37.307776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7c298fe9-1ac4-4bae-a054-4a72b4daa68e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:37.316741 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0e3f5969-1cd8-4ac3-a41e-3650dc77ace5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:37.317684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0e3f5969-1cd8-4ac3-a41e-3650dc77ace5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:37.318718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c298fe9-1ac4-4bae-a054-4a72b4daa68e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:37.319670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c298fe9-1ac4-4bae-a054-4a72b4daa68e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:37.320844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c298fe9-1ac4-4bae-a054-4a72b4daa68e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:37.321567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c298fe9-1ac4-4bae-a054-4a72b4daa68e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:37.328180 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e3f5969-1cd8-4ac3-a41e-3650dc77ace5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:37.328633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e3f5969-1cd8-4ac3-a41e-3650dc77ace5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:37.329372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e3f5969-1cd8-4ac3-a41e-3650dc77ace5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:37.329796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e3f5969-1cd8-4ac3-a41e-3650dc77ace5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:37.515846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7c298fe9-1ac4-4bae-a054-4a72b4daa68e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:37.520275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0e3f5969-1cd8-4ac3-a41e-3650dc77ace5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:37.525506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7c298fe9-1ac4-4bae-a054-4a72b4daa68e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.225317 May 25 16:39:37.526062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 746/1514] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:37 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:37.528912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0e3f5969-1cd8-4ac3-a41e-3650dc77ace5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.219170 May 25 16:39:37.529760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 769/1515] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:37 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:37.530748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e247b990-cfad-47ce-8284-d2d3e97f2286 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:37.531379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e247b990-cfad-47ce-8284-d2d3e97f2286 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:37.534260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:37.534850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:37.537406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e247b990-cfad-47ce-8284-d2d3e97f2286 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:37.537810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e247b990-cfad-47ce-8284-d2d3e97f2286 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:37.538599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e247b990-cfad-47ce-8284-d2d3e97f2286 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:37.539088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e247b990-cfad-47ce-8284-d2d3e97f2286 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:37.541562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:37.542202 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:37.544505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:37.544505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:37.622597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e247b990-cfad-47ce-8284-d2d3e97f2286 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:37.631139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e247b990-cfad-47ce-8284-d2d3e97f2286 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.104102 May 25 16:39:37.631776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 747/1516] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:37 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:37.694456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:37.703599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:37.705143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:37.706301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b0927c00-9c4b-4f65-8547-9b5386f363a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.175748 May 25 16:39:37.706859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 770/1517] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:37 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:38.552459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c3a33132-94b0-417f-8b73-40a1f99a935b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:38.553360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c3a33132-94b0-417f-8b73-40a1f99a935b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:38.554036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-86c778b1-a3d3-418a-9b05-103d416d01cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:38.554949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-86c778b1-a3d3-418a-9b05-103d416d01cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:38.563970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3a33132-94b0-417f-8b73-40a1f99a935b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:38.564653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3a33132-94b0-417f-8b73-40a1f99a935b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:38.564885 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86c778b1-a3d3-418a-9b05-103d416d01cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:38.565579 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86c778b1-a3d3-418a-9b05-103d416d01cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:38.565798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3a33132-94b0-417f-8b73-40a1f99a935b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:38.566454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3a33132-94b0-417f-8b73-40a1f99a935b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:38.566731 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86c778b1-a3d3-418a-9b05-103d416d01cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:38.567446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86c778b1-a3d3-418a-9b05-103d416d01cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:38.760033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-86c778b1-a3d3-418a-9b05-103d416d01cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:38.762201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:38.762901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c3a33132-94b0-417f-8b73-40a1f99a935b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:38.769376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-86c778b1-a3d3-418a-9b05-103d416d01cd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221179 May 25 16:39:38.769934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 771/1518] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:38 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:38.771994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c3a33132-94b0-417f-8b73-40a1f99a935b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.225481 May 25 16:39:38.772735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 748/1519] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:38 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:38.774618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-377965ff-ef32-4e4f-8ea1-6bd65096f51e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:38.781790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-377965ff-ef32-4e4f-8ea1-6bd65096f51e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:38.781957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:38.781957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:38.782117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-377965ff-ef32-4e4f-8ea1-6bd65096f51e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:38.782117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-377965ff-ef32-4e4f-8ea1-6bd65096f51e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:38.782537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-377965ff-ef32-4e4f-8ea1-6bd65096f51e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:38.782990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-377965ff-ef32-4e4f-8ea1-6bd65096f51e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:38.784194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:38.784645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:38.785400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:38.785838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:38.852479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-377965ff-ef32-4e4f-8ea1-6bd65096f51e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:38.861037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-377965ff-ef32-4e4f-8ea1-6bd65096f51e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090138 May 25 16:39:38.861649 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 772/1520] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:38 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:38.943276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:38.951914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:38.953423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:38.954583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fae0709b-e6a9-4e0d-a5c6-23f54a2c180a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.181064 May 25 16:39:38.955114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 749/1521] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:38 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:39.793548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-475b9317-8e7c-4461-89d2-7b2bf11f4fdd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:39.794467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8ef5bde1-6293-41c7-809b-46a286ca8d52 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:39.795082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-475b9317-8e7c-4461-89d2-7b2bf11f4fdd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:39.795318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8ef5bde1-6293-41c7-809b-46a286ca8d52 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:39.800444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=97 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:39.804471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-475b9317-8e7c-4461-89d2-7b2bf11f4fdd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:39.804927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-475b9317-8e7c-4461-89d2-7b2bf11f4fdd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:39.805449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ef5bde1-6293-41c7-809b-46a286ca8d52 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:39.805697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-475b9317-8e7c-4461-89d2-7b2bf11f4fdd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:39.805941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ef5bde1-6293-41c7-809b-46a286ca8d52 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:39.806135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-475b9317-8e7c-4461-89d2-7b2bf11f4fdd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:39.806706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ef5bde1-6293-41c7-809b-46a286ca8d52 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:39.807138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ef5bde1-6293-41c7-809b-46a286ca8d52 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:40.005790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8ef5bde1-6293-41c7-809b-46a286ca8d52 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:40.008835 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-475b9317-8e7c-4461-89d2-7b2bf11f4fdd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:40.014001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8ef5bde1-6293-41c7-809b-46a286ca8d52 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.227293 May 25 16:39:40.014705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 750/1522] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:39 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:40.019191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d0153f4d-8c5e-4d53-b888-1a6739a9c836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:40.019860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d0153f4d-8c5e-4d53-b888-1a6739a9c836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:40.022947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-475b9317-8e7c-4461-89d2-7b2bf11f4fdd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.235968 May 25 16:39:40.023862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 773/1523] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:39 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:40.026225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0153f4d-8c5e-4d53-b888-1a6739a9c836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:40.026746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0153f4d-8c5e-4d53-b888-1a6739a9c836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:40.027619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0153f4d-8c5e-4d53-b888-1a6739a9c836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:40.028116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0153f4d-8c5e-4d53-b888-1a6739a9c836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:40.029749 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:40.030605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:40.038367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:40.038824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:40.039903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:40.040345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:40.100014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d0153f4d-8c5e-4d53-b888-1a6739a9c836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:40.108574 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d0153f4d-8c5e-4d53-b888-1a6739a9c836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092774 May 25 16:39:40.109146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 751/1524] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:40 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:40.195645 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:40.205848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:40.207659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:40.209133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a94d045c-b8d2-47d8-bc85-0076b1fdff47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.184198 May 25 16:39:40.209808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 774/1525] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:40 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:41.034342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1a34f1d8-c39d-42fe-bd45-6f9cbdb4e69c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:41.035253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1a34f1d8-c39d-42fe-bd45-6f9cbdb4e69c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:41.043415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f4f1f765-ea3f-4585-b07b-dbbbf3a0ec7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:41.044340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f4f1f765-ea3f-4585-b07b-dbbbf3a0ec7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:41.044982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a34f1d8-c39d-42fe-bd45-6f9cbdb4e69c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:41.045639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a34f1d8-c39d-42fe-bd45-6f9cbdb4e69c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:41.046719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a34f1d8-c39d-42fe-bd45-6f9cbdb4e69c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:41.047353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1a34f1d8-c39d-42fe-bd45-6f9cbdb4e69c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:41.054436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f4f1f765-ea3f-4585-b07b-dbbbf3a0ec7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:41.054911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f4f1f765-ea3f-4585-b07b-dbbbf3a0ec7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:41.055702 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f4f1f765-ea3f-4585-b07b-dbbbf3a0ec7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:41.056134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f4f1f765-ea3f-4585-b07b-dbbbf3a0ec7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:41.248266 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1a34f1d8-c39d-42fe-bd45-6f9cbdb4e69c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:41.251582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f4f1f765-ea3f-4585-b07b-dbbbf3a0ec7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:41.256778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1a34f1d8-c39d-42fe-bd45-6f9cbdb4e69c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.228015 May 25 16:39:41.257358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 752/1526] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:41 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:41.260266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f4f1f765-ea3f-4585-b07b-dbbbf3a0ec7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222780 May 25 16:39:41.261389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 775/1527] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:41 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:41.261753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b7f56f55-c560-4291-901f-82a6cb3d0809 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:41.262346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b7f56f55-c560-4291-901f-82a6cb3d0809 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:41.266905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:41.267638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:41.273035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7f56f55-c560-4291-901f-82a6cb3d0809 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:41.273477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7f56f55-c560-4291-901f-82a6cb3d0809 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:41.274158 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:41.274326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7f56f55-c560-4291-901f-82a6cb3d0809 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:41.274584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:41.274744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7f56f55-c560-4291-901f-82a6cb3d0809 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:41.275408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:41.275842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:41.351469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b7f56f55-c560-4291-901f-82a6cb3d0809 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:41.359502 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b7f56f55-c560-4291-901f-82a6cb3d0809 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.101202 May 25 16:39:41.360113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 753/1528] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:41 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:41.448182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:41.458277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:41.459850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:41.461017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-838ca93e-a49b-45bd-9fd1-b373c4fcf363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.197767 May 25 16:39:41.461603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 776/1529] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:42.274254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-eb3115d6-d2f2-4551-ad70-5dfc5dd9a238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:42.274985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-eb3115d6-d2f2-4551-ad70-5dfc5dd9a238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:42.277493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-302432c5-9270-4534-aee8-00d7b1a6d361 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:42.278107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-302432c5-9270-4534-aee8-00d7b1a6d361 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:42.282272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb3115d6-d2f2-4551-ad70-5dfc5dd9a238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:42.282724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb3115d6-d2f2-4551-ad70-5dfc5dd9a238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:42.283531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb3115d6-d2f2-4551-ad70-5dfc5dd9a238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:42.283966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb3115d6-d2f2-4551-ad70-5dfc5dd9a238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:42.284672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-302432c5-9270-4534-aee8-00d7b1a6d361 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:42.285120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-302432c5-9270-4534-aee8-00d7b1a6d361 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:42.285935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-302432c5-9270-4534-aee8-00d7b1a6d361 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:42.286390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-302432c5-9270-4534-aee8-00d7b1a6d361 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:42.480273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-302432c5-9270-4534-aee8-00d7b1a6d361 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:42.483558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eb3115d6-d2f2-4551-ad70-5dfc5dd9a238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:42.488597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-302432c5-9270-4534-aee8-00d7b1a6d361 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.214878 May 25 16:39:42.489181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 777/1530] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:42 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:42.491868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-eb3115d6-d2f2-4551-ad70-5dfc5dd9a238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.221745 May 25 16:39:42.492602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 754/1531] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:42 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:42.493696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7a2be4e7-821a-4400-b328-34983c4d8793 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:42.494294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7a2be4e7-821a-4400-b328-34983c4d8793 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:42.496838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:42.497362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:42.500346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a2be4e7-821a-4400-b328-34983c4d8793 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:42.500797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a2be4e7-821a-4400-b328-34983c4d8793 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:42.501561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a2be4e7-821a-4400-b328-34983c4d8793 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:42.501993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a2be4e7-821a-4400-b328-34983c4d8793 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:42.503517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:42.503963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:42.504713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:42.505182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:42.572952 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7a2be4e7-821a-4400-b328-34983c4d8793 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:42.581123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7a2be4e7-821a-4400-b328-34983c4d8793 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090885 May 25 16:39:42.581691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 778/1532] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:42 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:42.661752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:42.670759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:42.672317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:42.673462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b0647ab3-749a-4a2c-a634-40eb32fc6c3c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.180115 May 25 16:39:42.674012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 755/1533] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:43.514351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-373c6ead-d815-46f6-bc2c-0757a8a1db2b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:43.515242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-373c6ead-d815-46f6-bc2c-0757a8a1db2b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:43.520280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-51c1680c-eafa-41e3-8e48-170a02244b30 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:43.521494 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-51c1680c-eafa-41e3-8e48-170a02244b30 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:43.525997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-373c6ead-d815-46f6-bc2c-0757a8a1db2b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:43.526762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-373c6ead-d815-46f6-bc2c-0757a8a1db2b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:43.528088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-373c6ead-d815-46f6-bc2c-0757a8a1db2b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:43.528808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-373c6ead-d815-46f6-bc2c-0757a8a1db2b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:43.531879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51c1680c-eafa-41e3-8e48-170a02244b30 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:43.532636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51c1680c-eafa-41e3-8e48-170a02244b30 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:43.533925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51c1680c-eafa-41e3-8e48-170a02244b30 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:43.534668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51c1680c-eafa-41e3-8e48-170a02244b30 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:43.723102 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-373c6ead-d815-46f6-bc2c-0757a8a1db2b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:43.726191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51c1680c-eafa-41e3-8e48-170a02244b30 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:43.731519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-373c6ead-d815-46f6-bc2c-0757a8a1db2b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.223966 May 25 16:39:43.732074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 779/1534] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:43 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:43.734363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-51c1680c-eafa-41e3-8e48-170a02244b30 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221229 May 25 16:39:43.736110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 756/1535] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:43 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:43.736846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7b94007a-e56f-4b83-80b9-a8dbce18656f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:43.737447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7b94007a-e56f-4b83-80b9-a8dbce18656f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:43.743293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:43.743571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7b94007a-e56f-4b83-80b9-a8dbce18656f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:43.743898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:43.744040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7b94007a-e56f-4b83-80b9-a8dbce18656f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:43.744923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7b94007a-e56f-4b83-80b9-a8dbce18656f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:43.745360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7b94007a-e56f-4b83-80b9-a8dbce18656f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:43.750184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:43.750629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:43.751431 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:43.751861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:43.813806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7b94007a-e56f-4b83-80b9-a8dbce18656f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:43.822392 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7b94007a-e56f-4b83-80b9-a8dbce18656f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089341 May 25 16:39:43.822964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 780/1536] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:43 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:43.896070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:43.905089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:43.906745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:43.907953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c09e4234-aecd-4649-bacd-9e7e82473723 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.172045 May 25 16:39:43.908522 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 757/1537] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:44.753168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-939cc648-4589-4696-ba1b-666e5b73e0d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:44.754122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-939cc648-4589-4696-ba1b-666e5b73e0d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:44.755389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a0a21f7f-e142-412c-b724-e75fb67221cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:44.755988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a0a21f7f-e142-412c-b724-e75fb67221cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:44.761628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-939cc648-4589-4696-ba1b-666e5b73e0d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:44.762067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-939cc648-4589-4696-ba1b-666e5b73e0d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:44.762795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0a21f7f-e142-412c-b724-e75fb67221cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:44.762982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-939cc648-4589-4696-ba1b-666e5b73e0d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:44.763269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0a21f7f-e142-412c-b724-e75fb67221cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:44.763402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-939cc648-4589-4696-ba1b-666e5b73e0d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:44.764019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0a21f7f-e142-412c-b724-e75fb67221cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:44.764439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0a21f7f-e142-412c-b724-e75fb67221cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:44.967443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-939cc648-4589-4696-ba1b-666e5b73e0d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:44.970420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a0a21f7f-e142-412c-b724-e75fb67221cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:44.975349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-939cc648-4589-4696-ba1b-666e5b73e0d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.228671 May 25 16:39:44.976502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 781/1538] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:44 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:44.980931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a8d879d1-2162-48e7-b809-a79e434f071a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:44.981520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a8d879d1-2162-48e7-b809-a79e434f071a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:44.987102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a0a21f7f-e142-412c-b724-e75fb67221cb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.237975 May 25 16:39:44.987344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8d879d1-2162-48e7-b809-a79e434f071a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:44.987787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8d879d1-2162-48e7-b809-a79e434f071a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:44.988537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8d879d1-2162-48e7-b809-a79e434f071a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:44.989208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 758/1539] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:44 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:44.989339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8d879d1-2162-48e7-b809-a79e434f071a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:44.993740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:44.994374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:45.002620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:45.003141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:45.004003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:45.004496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:45.091662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a8d879d1-2162-48e7-b809-a79e434f071a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:45.099796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a8d879d1-2162-48e7-b809-a79e434f071a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.122314 May 25 16:39:45.100380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 782/1540] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:44 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:45.157673 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:45.166472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:45.167994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:45.169158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-696b0cbf-b447-45e4-b2b0-97f92c1afbe3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.179203 May 25 16:39:45.169719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 759/1541] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:44 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:45.997278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b1498ef3-6c2e-4e88-9896-ef31d8d9755e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:45.998235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b1498ef3-6c2e-4e88-9896-ef31d8d9755e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:46.009279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b1498ef3-6c2e-4e88-9896-ef31d8d9755e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:46.009584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-badf67b7-3db2-4c0c-91a5-0c0612203c52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:46.010253 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b1498ef3-6c2e-4e88-9896-ef31d8d9755e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:46.010444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-badf67b7-3db2-4c0c-91a5-0c0612203c52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:46.011256 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b1498ef3-6c2e-4e88-9896-ef31d8d9755e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:46.012026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b1498ef3-6c2e-4e88-9896-ef31d8d9755e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:46.021323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-badf67b7-3db2-4c0c-91a5-0c0612203c52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:46.021751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-badf67b7-3db2-4c0c-91a5-0c0612203c52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:46.022515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-badf67b7-3db2-4c0c-91a5-0c0612203c52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:46.022961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-badf67b7-3db2-4c0c-91a5-0c0612203c52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:46.207485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b1498ef3-6c2e-4e88-9896-ef31d8d9755e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:46.211387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-badf67b7-3db2-4c0c-91a5-0c0612203c52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:46.217381 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b1498ef3-6c2e-4e88-9896-ef31d8d9755e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.226595 May 25 16:39:46.217973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 783/1542] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:45 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:46.222662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b5590ae7-0ac3-4118-b536-b437100bb0c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:46.223260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b5590ae7-0ac3-4118-b536-b437100bb0c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:46.224998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=98 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:46.226439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-badf67b7-3db2-4c0c-91a5-0c0612203c52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.223437 May 25 16:39:46.230158 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5590ae7-0ac3-4118-b536-b437100bb0c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:46.230605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5590ae7-0ac3-4118-b536-b437100bb0c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:46.231390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5590ae7-0ac3-4118-b536-b437100bb0c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:46.231821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5590ae7-0ac3-4118-b536-b437100bb0c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:46.234565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 760/1543] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:46 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:46.239506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:46.240111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:46.246548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:46.247007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:46.247864 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:46.248262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:46.295643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b5590ae7-0ac3-4118-b536-b437100bb0c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:46.303758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b5590ae7-0ac3-4118-b536-b437100bb0c6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084703 May 25 16:39:46.304338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 784/1544] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:46 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:46.392064 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:46.395278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:46.406011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:46.408635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:46.410557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9e83be72-e632-4b21-8392-a5cae03c3bf7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.174556 May 25 16:39:46.411494 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 761/1545] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:46 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:47.242262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2c44bfee-a40a-4ed1-9615-3ab6c0019a65 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:47.244690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2c44bfee-a40a-4ed1-9615-3ab6c0019a65 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:47.248217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-21b81645-eb29-4e9e-ace2-0c882be14226 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:47.249312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-21b81645-eb29-4e9e-ace2-0c882be14226 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:47.255682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c44bfee-a40a-4ed1-9615-3ab6c0019a65 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:47.256122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c44bfee-a40a-4ed1-9615-3ab6c0019a65 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:47.256882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c44bfee-a40a-4ed1-9615-3ab6c0019a65 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:47.257430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c44bfee-a40a-4ed1-9615-3ab6c0019a65 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:47.258167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21b81645-eb29-4e9e-ace2-0c882be14226 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:47.258614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21b81645-eb29-4e9e-ace2-0c882be14226 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:47.259419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21b81645-eb29-4e9e-ace2-0c882be14226 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:47.259914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21b81645-eb29-4e9e-ace2-0c882be14226 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:47.460973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2c44bfee-a40a-4ed1-9615-3ab6c0019a65 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:47.467750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21b81645-eb29-4e9e-ace2-0c882be14226 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:47.469298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2c44bfee-a40a-4ed1-9615-3ab6c0019a65 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.233514 May 25 16:39:47.469843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 785/1546] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:47 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:47.474232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-13a95f54-561f-4e5a-a782-7b2337f6cbfe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:47.474829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-13a95f54-561f-4e5a-a782-7b2337f6cbfe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:47.476029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-21b81645-eb29-4e9e-ace2-0c882be14226 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.234680 May 25 16:39:47.476797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 762/1547] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:47 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:47.483914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:47.484554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:47.484927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13a95f54-561f-4e5a-a782-7b2337f6cbfe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:47.485442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13a95f54-561f-4e5a-a782-7b2337f6cbfe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:47.486447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13a95f54-561f-4e5a-a782-7b2337f6cbfe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:47.486885 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13a95f54-561f-4e5a-a782-7b2337f6cbfe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:47.490571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:47.491023 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:47.491823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:47.492287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:47.564276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-13a95f54-561f-4e5a-a782-7b2337f6cbfe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:47.572424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-13a95f54-561f-4e5a-a782-7b2337f6cbfe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.101629 May 25 16:39:47.572999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 786/1548] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:47 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:47.649632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:47.662529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:47.665241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:47.667069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-21016775-c805-4680-9c78-a37bdfcbae7b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.189290 May 25 16:39:47.667960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 763/1549] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:48.490207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7168311e-3ebf-4f0e-9291-b32fc4aa7eac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:48.490765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7168311e-3ebf-4f0e-9291-b32fc4aa7eac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:48.493922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4aae6829-46bc-4ce7-ac37-cac2311f9b31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:48.494526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4aae6829-46bc-4ce7-ac37-cac2311f9b31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:48.497757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7168311e-3ebf-4f0e-9291-b32fc4aa7eac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:48.498208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7168311e-3ebf-4f0e-9291-b32fc4aa7eac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:48.498971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7168311e-3ebf-4f0e-9291-b32fc4aa7eac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:48.499441 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7168311e-3ebf-4f0e-9291-b32fc4aa7eac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:48.501493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4aae6829-46bc-4ce7-ac37-cac2311f9b31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:48.501918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4aae6829-46bc-4ce7-ac37-cac2311f9b31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:48.502687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4aae6829-46bc-4ce7-ac37-cac2311f9b31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:48.503108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4aae6829-46bc-4ce7-ac37-cac2311f9b31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:48.700190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7168311e-3ebf-4f0e-9291-b32fc4aa7eac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:48.703180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4aae6829-46bc-4ce7-ac37-cac2311f9b31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:48.708476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7168311e-3ebf-4f0e-9291-b32fc4aa7eac tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.224520 May 25 16:39:48.709126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 787/1550] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:48 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:48.712493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4aae6829-46bc-4ce7-ac37-cac2311f9b31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222298 May 25 16:39:48.713299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 764/1551] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:48 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:48.713542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-948e82f0-f657-4cea-9568-57a5c19f36d0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:48.714131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-948e82f0-f657-4cea-9568-57a5c19f36d0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:48.720087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-948e82f0-f657-4cea-9568-57a5c19f36d0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:48.720539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-948e82f0-f657-4cea-9568-57a5c19f36d0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:48.721301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-948e82f0-f657-4cea-9568-57a5c19f36d0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:48.721737 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-948e82f0-f657-4cea-9568-57a5c19f36d0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:48.724799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:48.725378 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:48.731349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:48.731812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:48.732563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:48.732990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:48.736568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:48.807271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-948e82f0-f657-4cea-9568-57a5c19f36d0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:48.815729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-948e82f0-f657-4cea-9568-57a5c19f36d0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.105654 May 25 16:39:48.816316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 788/1552] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:48 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:48.872931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:48.886527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:48.888935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:48.890792 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-de7d6bce-24bf-47c5-a511-0c2a237d6e34 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.169542 May 25 16:39:48.891812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 765/1553] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:49.730048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e95c347e-1543-4870-a33d-9f7c08e134af tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:49.731196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e95c347e-1543-4870-a33d-9f7c08e134af tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:49.733954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c4c362cc-cc8c-45cb-8dea-194a90c9873a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:49.734938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c4c362cc-cc8c-45cb-8dea-194a90c9873a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:49.741160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e95c347e-1543-4870-a33d-9f7c08e134af tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:49.741604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e95c347e-1543-4870-a33d-9f7c08e134af tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:49.742360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e95c347e-1543-4870-a33d-9f7c08e134af tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:49.742783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e95c347e-1543-4870-a33d-9f7c08e134af tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:49.743477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4c362cc-cc8c-45cb-8dea-194a90c9873a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:49.743927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4c362cc-cc8c-45cb-8dea-194a90c9873a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:49.744660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4c362cc-cc8c-45cb-8dea-194a90c9873a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:49.745094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4c362cc-cc8c-45cb-8dea-194a90c9873a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:49.935069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e95c347e-1543-4870-a33d-9f7c08e134af tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:49.938265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c4c362cc-cc8c-45cb-8dea-194a90c9873a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:49.943447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e95c347e-1543-4870-a33d-9f7c08e134af tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.219937 May 25 16:39:49.944032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 789/1554] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:49 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:49.946975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c4c362cc-cc8c-45cb-8dea-194a90c9873a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.219911 May 25 16:39:49.947755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 766/1555] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:49 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:49.952118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:49.952764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:49.954087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1253cda7-2861-4f64-b4cc-850f42c768df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:49.954903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1253cda7-2861-4f64-b4cc-850f42c768df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:49.958906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:49.959438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:49.960157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:49.960590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:49.960941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1253cda7-2861-4f64-b4cc-850f42c768df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:49.961382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1253cda7-2861-4f64-b4cc-850f42c768df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:49.962154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1253cda7-2861-4f64-b4cc-850f42c768df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:49.962577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1253cda7-2861-4f64-b4cc-850f42c768df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:50.053627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1253cda7-2861-4f64-b4cc-850f42c768df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:50.061906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1253cda7-2861-4f64-b4cc-850f42c768df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.116923 May 25 16:39:50.062485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 790/1556] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:49 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:50.118943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:50.127605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:50.129111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:50.130378 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-eb8ee475-d7d0-44fa-9721-e337f5a31f32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.181834 May 25 16:39:50.130915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 767/1557] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:50.965878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9c067eed-3e21-48a3-9573-10004bf076d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:50.967086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9c067eed-3e21-48a3-9573-10004bf076d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:50.968091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4323c988-65ef-459b-a931-ec726abaa4a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:50.969187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4323c988-65ef-459b-a931-ec726abaa4a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:50.976844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9c067eed-3e21-48a3-9573-10004bf076d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:50.977278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9c067eed-3e21-48a3-9573-10004bf076d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:50.977810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4323c988-65ef-459b-a931-ec726abaa4a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:50.978059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9c067eed-3e21-48a3-9573-10004bf076d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:50.978284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4323c988-65ef-459b-a931-ec726abaa4a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:50.978459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9c067eed-3e21-48a3-9573-10004bf076d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:50.979051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4323c988-65ef-459b-a931-ec726abaa4a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:50.979515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4323c988-65ef-459b-a931-ec726abaa4a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:51.182730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4323c988-65ef-459b-a931-ec726abaa4a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:51.186357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9c067eed-3e21-48a3-9573-10004bf076d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:51.191365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4323c988-65ef-459b-a931-ec726abaa4a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.229851 May 25 16:39:51.192197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 768/1558] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:50 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:51.194777 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9c067eed-3e21-48a3-9573-10004bf076d8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.236440 May 25 16:39:51.195556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 791/1559] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:50 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:51.196617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f535463b-ba8f-483b-88f7-3b639da170e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:51.197195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f535463b-ba8f-483b-88f7-3b639da170e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:51.199981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:51.200566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:51.203750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f535463b-ba8f-483b-88f7-3b639da170e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:51.204231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f535463b-ba8f-483b-88f7-3b639da170e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:51.204978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f535463b-ba8f-483b-88f7-3b639da170e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:51.205414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f535463b-ba8f-483b-88f7-3b639da170e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:51.208911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:51.210662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:51.210662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:51.210662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:51.274170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f535463b-ba8f-483b-88f7-3b639da170e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:51.282212 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f535463b-ba8f-483b-88f7-3b639da170e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089151 May 25 16:39:51.282781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 769/1560] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:51 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:51.367526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:51.376546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:51.378129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:51.379364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-401e338d-5e3f-4b21-bcd6-f86239c2afe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.183007 May 25 16:39:51.379917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 792/1561] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:52.217142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-49f86fc5-ac7f-493a-8cdd-fa9ad5a38e88 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:52.218046 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-49f86fc5-ac7f-493a-8cdd-fa9ad5a38e88 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:52.222842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-23d7a36f-43dc-47c5-97bf-99be50e12d45 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:52.224149 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-23d7a36f-43dc-47c5-97bf-99be50e12d45 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:52.227840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49f86fc5-ac7f-493a-8cdd-fa9ad5a38e88 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:52.228683 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49f86fc5-ac7f-493a-8cdd-fa9ad5a38e88 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:52.229110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49f86fc5-ac7f-493a-8cdd-fa9ad5a38e88 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:52.229539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49f86fc5-ac7f-493a-8cdd-fa9ad5a38e88 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:52.232507 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23d7a36f-43dc-47c5-97bf-99be50e12d45 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:52.233006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23d7a36f-43dc-47c5-97bf-99be50e12d45 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:52.233906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23d7a36f-43dc-47c5-97bf-99be50e12d45 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:52.234304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23d7a36f-43dc-47c5-97bf-99be50e12d45 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:52.421285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-23d7a36f-43dc-47c5-97bf-99be50e12d45 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:52.424121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-49f86fc5-ac7f-493a-8cdd-fa9ad5a38e88 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:52.429664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-23d7a36f-43dc-47c5-97bf-99be50e12d45 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.214262 May 25 16:39:52.430315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 793/1562] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:52 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:52.432199 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-49f86fc5-ac7f-493a-8cdd-fa9ad5a38e88 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221630 May 25 16:39:52.440115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 770/1563] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:52 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:52.440312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f60b4d6a-767f-4285-9391-74a51bad9b39 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:52.440312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f60b4d6a-767f-4285-9391-74a51bad9b39 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:52.440521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:52.440521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:52.443469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f60b4d6a-767f-4285-9391-74a51bad9b39 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:52.443658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:52.443912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:52.444086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f60b4d6a-767f-4285-9391-74a51bad9b39 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:52.444653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:52.444954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f60b4d6a-767f-4285-9391-74a51bad9b39 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:52.445070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:52.445564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f60b4d6a-767f-4285-9391-74a51bad9b39 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:52.534283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f60b4d6a-767f-4285-9391-74a51bad9b39 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:52.543889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f60b4d6a-767f-4285-9391-74a51bad9b39 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.112191 May 25 16:39:52.544475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 794/1564] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:52 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:52.605947 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:52.614534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:52.616070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:52.617232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-302d3e3d-4340-4c35-9f89-9ffced7e799d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.183587 May 25 16:39:52.617886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 771/1565] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:53.450601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-39575615-6348-4fac-ba55-4994ea665753 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:53.451289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-39575615-6348-4fac-ba55-4994ea665753 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:53.451452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-00ae1b08-015c-46ce-9d90-33c758f25fbb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:53.452093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-00ae1b08-015c-46ce-9d90-33c758f25fbb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:53.457950 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39575615-6348-4fac-ba55-4994ea665753 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:53.458398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39575615-6348-4fac-ba55-4994ea665753 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:53.458922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00ae1b08-015c-46ce-9d90-33c758f25fbb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:53.459132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39575615-6348-4fac-ba55-4994ea665753 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:53.459395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00ae1b08-015c-46ce-9d90-33c758f25fbb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:53.459662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39575615-6348-4fac-ba55-4994ea665753 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:53.460159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00ae1b08-015c-46ce-9d90-33c758f25fbb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:53.460595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00ae1b08-015c-46ce-9d90-33c758f25fbb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:53.657136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-39575615-6348-4fac-ba55-4994ea665753 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:53.659935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-00ae1b08-015c-46ce-9d90-33c758f25fbb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:53.666115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-39575615-6348-4fac-ba55-4994ea665753 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.221120 May 25 16:39:53.666780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 795/1566] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:53 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:53.669115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-00ae1b08-015c-46ce-9d90-33c758f25fbb tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222451 May 25 16:39:53.678823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 772/1567] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:53 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:53.679013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a8c50448-3770-496a-baf9-33827e855abe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:53.679013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a8c50448-3770-496a-baf9-33827e855abe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:53.679252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:53.679252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:53.679435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8c50448-3770-496a-baf9-33827e855abe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:53.679435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8c50448-3770-496a-baf9-33827e855abe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:53.679435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8c50448-3770-496a-baf9-33827e855abe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:53.679832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8c50448-3770-496a-baf9-33827e855abe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:53.680681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:53.681134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:53.681951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:53.682387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:53.764076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a8c50448-3770-496a-baf9-33827e855abe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:53.773291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a8c50448-3770-496a-baf9-33827e855abe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.105308 May 25 16:39:53.773872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 796/1568] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:53 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:53.835644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:53.844267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:53.845757 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:53.846919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-98d5d299-00fd-41e1-a45d-a102df4b8bed tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.176313 May 25 16:39:53.847509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 773/1569] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:53 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:54.686846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-29244c36-c1f8-466c-9ed6-a62a6c1314f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:54.687632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-29244c36-c1f8-466c-9ed6-a62a6c1314f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:54.688397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-42b9ab73-e159-4c30-9b45-7603f2a537b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:54.689066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-42b9ab73-e159-4c30-9b45-7603f2a537b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:54.695590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29244c36-c1f8-466c-9ed6-a62a6c1314f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:54.696086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29244c36-c1f8-466c-9ed6-a62a6c1314f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:54.696553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-42b9ab73-e159-4c30-9b45-7603f2a537b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:54.696947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29244c36-c1f8-466c-9ed6-a62a6c1314f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:54.697144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-42b9ab73-e159-4c30-9b45-7603f2a537b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:54.697420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-29244c36-c1f8-466c-9ed6-a62a6c1314f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:54.697877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-42b9ab73-e159-4c30-9b45-7603f2a537b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:54.698414 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-42b9ab73-e159-4c30-9b45-7603f2a537b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:54.900441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-42b9ab73-e159-4c30-9b45-7603f2a537b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:54.903376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-29244c36-c1f8-466c-9ed6-a62a6c1314f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:54.908692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-42b9ab73-e159-4c30-9b45-7603f2a537b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.225385 May 25 16:39:54.909260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 774/1570] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:54 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:54.911594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-29244c36-c1f8-466c-9ed6-a62a6c1314f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.231180 May 25 16:39:54.916212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 797/1571] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:54 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:54.916383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-35a04181-0995-4728-997d-82128bc6e727 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:54.916383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-35a04181-0995-4728-997d-82128bc6e727 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:54.917267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:54.917877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:54.921922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35a04181-0995-4728-997d-82128bc6e727 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:54.922588 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35a04181-0995-4728-997d-82128bc6e727 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:54.923370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35a04181-0995-4728-997d-82128bc6e727 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:54.923794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35a04181-0995-4728-997d-82128bc6e727 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:54.923959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:54.924331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:54.925097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:54.925533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:54.991416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-35a04181-0995-4728-997d-82128bc6e727 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:54.999628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-35a04181-0995-4728-997d-82128bc6e727 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089394 May 25 16:39:55.000194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 775/1572] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:54 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:55.074697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:55.083654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:55.085196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:55.086360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-559691ee-76ec-4f28-a177-ef09488aec50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.172991 May 25 16:39:55.086898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 798/1573] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:55.930307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-89ab296c-22e2-4673-a5c6-2715c82310e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:55.931565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-89ab296c-22e2-4673-a5c6-2715c82310e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:55.933271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5c05dfdb-3e16-4716-80c8-62e3f3be00e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:55.934278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5c05dfdb-3e16-4716-80c8-62e3f3be00e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:55.942677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89ab296c-22e2-4673-a5c6-2715c82310e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:55.943482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89ab296c-22e2-4673-a5c6-2715c82310e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:55.944736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89ab296c-22e2-4673-a5c6-2715c82310e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:55.944976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c05dfdb-3e16-4716-80c8-62e3f3be00e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:55.945454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89ab296c-22e2-4673-a5c6-2715c82310e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:55.945684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c05dfdb-3e16-4716-80c8-62e3f3be00e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:55.946804 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c05dfdb-3e16-4716-80c8-62e3f3be00e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:55.947578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c05dfdb-3e16-4716-80c8-62e3f3be00e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:56.140486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5c05dfdb-3e16-4716-80c8-62e3f3be00e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:56.143798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-89ab296c-22e2-4673-a5c6-2715c82310e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:56.152679 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5c05dfdb-3e16-4716-80c8-62e3f3be00e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.225600 May 25 16:39:56.153373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 799/1574] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:55 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:56.155075 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-89ab296c-22e2-4673-a5c6-2715c82310e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.231169 May 25 16:39:56.155903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 776/1575] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:55 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:56.157828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-eb5ae44d-0d0c-4c5d-9b91-34bcb3fafae7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:56.158428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-eb5ae44d-0d0c-4c5d-9b91-34bcb3fafae7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:56.160231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:56.160958 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:56.164789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eb5ae44d-0d0c-4c5d-9b91-34bcb3fafae7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:56.165318 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eb5ae44d-0d0c-4c5d-9b91-34bcb3fafae7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:56.166168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eb5ae44d-0d0c-4c5d-9b91-34bcb3fafae7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:56.166607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eb5ae44d-0d0c-4c5d-9b91-34bcb3fafae7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:56.167400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:56.167847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:56.168614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:56.169166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:56.238384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-eb5ae44d-0d0c-4c5d-9b91-34bcb3fafae7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:56.246750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-eb5ae44d-0d0c-4c5d-9b91-34bcb3fafae7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092427 May 25 16:39:56.247327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 800/1576] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:56 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:56.321222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:56.329985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:56.331532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:56.332694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-42e00b49-afd4-4e4e-9961-ef86ea2492d3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.176025 May 25 16:39:56.333253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 777/1577] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:56 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:57.182794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ceba3053-b5c0-41b2-b6b6-8e66a060e610 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:57.183522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d2719367-267a-4483-951b-0bd9e7ed7693 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:57.183918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ceba3053-b5c0-41b2-b6b6-8e66a060e610 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:57.184017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d2719367-267a-4483-951b-0bd9e7ed7693 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:57.190572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ceba3053-b5c0-41b2-b6b6-8e66a060e610 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:57.190765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2719367-267a-4483-951b-0bd9e7ed7693 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:57.191026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ceba3053-b5c0-41b2-b6b6-8e66a060e610 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:57.191209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2719367-267a-4483-951b-0bd9e7ed7693 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:57.191827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ceba3053-b5c0-41b2-b6b6-8e66a060e610 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:57.191974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2719367-267a-4483-951b-0bd9e7ed7693 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:57.192271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ceba3053-b5c0-41b2-b6b6-8e66a060e610 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:57.192436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2719367-267a-4483-951b-0bd9e7ed7693 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:57.207719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:39:57.386759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ceba3053-b5c0-41b2-b6b6-8e66a060e610 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:57.392810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d2719367-267a-4483-951b-0bd9e7ed7693 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:57.395048 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ceba3053-b5c0-41b2-b6b6-8e66a060e610 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.218622 May 25 16:39:57.395635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 778/1578] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:57 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:57.400072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ed1478ca-3179-464f-be1e-c47360185dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:57.400661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ed1478ca-3179-464f-be1e-c47360185dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:57.401321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d2719367-267a-4483-951b-0bd9e7ed7693 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.226139 May 25 16:39:57.402065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 801/1579] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:57 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:57.406523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:57.406684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed1478ca-3179-464f-be1e-c47360185dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:57.407126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:57.407282 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed1478ca-3179-464f-be1e-c47360185dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:57.407910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed1478ca-3179-464f-be1e-c47360185dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:57.408340 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed1478ca-3179-464f-be1e-c47360185dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:57.413032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:57.413475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:57.414236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:57.414983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:57.494697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ed1478ca-3179-464f-be1e-c47360185dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:57.503070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ed1478ca-3179-464f-be1e-c47360185dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.106457 May 25 16:39:57.503679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 779/1580] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:57 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:57.565792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:57.574622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:57.576405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:57.577573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1d8551ed-2287-4f2c-90cc-b96b245409a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.174753 May 25 16:39:57.578136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 802/1581] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:58.421605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b29e33ff-b1e3-4a26-a228-2fc796bf5513 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:58.422289 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b29e33ff-b1e3-4a26-a228-2fc796bf5513 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:58.424067 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-88f169e2-7e44-4d7d-9ed3-97749ea67b92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:58.424883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-88f169e2-7e44-4d7d-9ed3-97749ea67b92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:58.429172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b29e33ff-b1e3-4a26-a228-2fc796bf5513 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:58.429768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b29e33ff-b1e3-4a26-a228-2fc796bf5513 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:58.430591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b29e33ff-b1e3-4a26-a228-2fc796bf5513 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:58.431001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88f169e2-7e44-4d7d-9ed3-97749ea67b92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:58.431197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b29e33ff-b1e3-4a26-a228-2fc796bf5513 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:58.431474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88f169e2-7e44-4d7d-9ed3-97749ea67b92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:58.432232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88f169e2-7e44-4d7d-9ed3-97749ea67b92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:58.432665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88f169e2-7e44-4d7d-9ed3-97749ea67b92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:58.621190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-88f169e2-7e44-4d7d-9ed3-97749ea67b92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:58.625824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b29e33ff-b1e3-4a26-a228-2fc796bf5513 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:58.629363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-88f169e2-7e44-4d7d-9ed3-97749ea67b92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.209897 May 25 16:39:58.629974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 803/1582] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:58 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:39:58.634365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b60f13b7-9628-4300-8c5a-ea064f902ff1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:58.634746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b29e33ff-b1e3-4a26-a228-2fc796bf5513 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.218503 May 25 16:39:58.634961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b60f13b7-9628-4300-8c5a-ea064f902ff1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:58.635630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 780/1583] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:58 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:58.640069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:58.640756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:58.640887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b60f13b7-9628-4300-8c5a-ea064f902ff1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:58.641205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b60f13b7-9628-4300-8c5a-ea064f902ff1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:58.641961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b60f13b7-9628-4300-8c5a-ea064f902ff1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:58.642393 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b60f13b7-9628-4300-8c5a-ea064f902ff1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:58.649679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:58.650131 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:58.650898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:58.651368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:58.714868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b60f13b7-9628-4300-8c5a-ea064f902ff1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:58.723288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b60f13b7-9628-4300-8c5a-ea064f902ff1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092329 May 25 16:39:58.723898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 804/1584] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:58 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:39:58.806515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:58.815251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:58.816802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:39:58.818003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b7701797-40ae-477b-bd50-c4962b9ad616 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.181512 May 25 16:39:58.818560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 781/1585] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:39:58 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:39:59.656433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-339b64e2-5545-4a56-8583-41762fe5fa7c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:59.657411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-339b64e2-5545-4a56-8583-41762fe5fa7c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:59.658423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d5376fa6-5f53-4003-abb1-8e8a3627674d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:39:59.659530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d5376fa6-5f53-4003-abb1-8e8a3627674d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:39:59.668410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-339b64e2-5545-4a56-8583-41762fe5fa7c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:59.668855 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-339b64e2-5545-4a56-8583-41762fe5fa7c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:59.669452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5376fa6-5f53-4003-abb1-8e8a3627674d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:59.669658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-339b64e2-5545-4a56-8583-41762fe5fa7c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:59.669908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5376fa6-5f53-4003-abb1-8e8a3627674d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:59.670058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-339b64e2-5545-4a56-8583-41762fe5fa7c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:59.670670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5376fa6-5f53-4003-abb1-8e8a3627674d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:39:59.671110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5376fa6-5f53-4003-abb1-8e8a3627674d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:39:59.672668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:00.106056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d5376fa6-5f53-4003-abb1-8e8a3627674d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:00.110083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-339b64e2-5545-4a56-8583-41762fe5fa7c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:00.115482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d5376fa6-5f53-4003-abb1-8e8a3627674d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.463205 May 25 16:40:00.125793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 782/1586] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:59 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:00.125998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-339b64e2-5545-4a56-8583-41762fe5fa7c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.468517 May 25 16:40:00.125998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 805/1587] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:39:59 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:00.126174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7dd32596-954a-48a9-8170-f48b7d23551b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:00.126174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7dd32596-954a-48a9-8170-f48b7d23551b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:00.126354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:00.126354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:00.130097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:00.130526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:00.130853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dd32596-954a-48a9-8170-f48b7d23551b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:00.131304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:00.131417 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dd32596-954a-48a9-8170-f48b7d23551b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:00.131732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:00.132086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dd32596-954a-48a9-8170-f48b7d23551b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:00.132519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dd32596-954a-48a9-8170-f48b7d23551b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:00.218935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7dd32596-954a-48a9-8170-f48b7d23551b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:00.227185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7dd32596-954a-48a9-8170-f48b7d23551b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.109913 May 25 16:40:00.227767 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 783/1588] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:00 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:00.285596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:00.294479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:00.296015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:00.297181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9a6dea96-21fa-4bfa-aa85-d69152d4e09b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.177390 May 25 16:40:00.297745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 806/1589] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:00 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:01.134261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ae45d73c-3bc8-4b72-af3c-3a417a0f7db3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:01.134870 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ae45d73c-3bc8-4b72-af3c-3a417a0f7db3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:01.135564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3007998f-f92d-45aa-a5f8-700a814a8257 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:01.136254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3007998f-f92d-45aa-a5f8-700a814a8257 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:01.143245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3007998f-f92d-45aa-a5f8-700a814a8257 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:01.143700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3007998f-f92d-45aa-a5f8-700a814a8257 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:01.144463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3007998f-f92d-45aa-a5f8-700a814a8257 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:01.144807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae45d73c-3bc8-4b72-af3c-3a417a0f7db3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:01.144981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3007998f-f92d-45aa-a5f8-700a814a8257 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:01.145234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae45d73c-3bc8-4b72-af3c-3a417a0f7db3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:01.145992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae45d73c-3bc8-4b72-af3c-3a417a0f7db3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:01.146450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae45d73c-3bc8-4b72-af3c-3a417a0f7db3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:01.345278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3007998f-f92d-45aa-a5f8-700a814a8257 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:01.348551 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ae45d73c-3bc8-4b72-af3c-3a417a0f7db3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:01.353541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3007998f-f92d-45aa-a5f8-700a814a8257 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.221788 May 25 16:40:01.354116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 807/1590] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:01 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:01.358724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6b283cb5-e637-466d-9600-bc679361c353 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:01.359374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6b283cb5-e637-466d-9600-bc679361c353 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:01.362697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ae45d73c-3bc8-4b72-af3c-3a417a0f7db3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.232291 May 25 16:40:01.363443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 784/1591] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:01 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:01.365792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b283cb5-e637-466d-9600-bc679361c353 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:01.366267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b283cb5-e637-466d-9600-bc679361c353 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:01.367019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b283cb5-e637-466d-9600-bc679361c353 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:01.367526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b283cb5-e637-466d-9600-bc679361c353 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:01.368048 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:01.368679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:01.374721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:01.375198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:01.375957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:01.376386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:01.437181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6b283cb5-e637-466d-9600-bc679361c353 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:01.446395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6b283cb5-e637-466d-9600-bc679361c353 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091204 May 25 16:40:01.446985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 808/1592] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:01 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:01.534127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:01.546347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:01.547881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:01.549031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-be5a6fe7-093c-4fe7-987d-aa8cd5d18a57 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.184655 May 25 16:40:01.549623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 785/1593] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:01 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:02.373970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f3a09077-08be-4a50-a8ab-d9c8aeee55b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:02.374718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f3a09077-08be-4a50-a8ab-d9c8aeee55b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:02.382791 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3a09077-08be-4a50-a8ab-d9c8aeee55b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:02.383424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3a09077-08be-4a50-a8ab-d9c8aeee55b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:02.383628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-665d8f54-8271-4456-a431-faa78b375dff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:02.384371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3a09077-08be-4a50-a8ab-d9c8aeee55b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:02.384546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-665d8f54-8271-4456-a431-faa78b375dff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:02.384911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3a09077-08be-4a50-a8ab-d9c8aeee55b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:02.393725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-665d8f54-8271-4456-a431-faa78b375dff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:02.394354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-665d8f54-8271-4456-a431-faa78b375dff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:02.395470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-665d8f54-8271-4456-a431-faa78b375dff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:02.396056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-665d8f54-8271-4456-a431-faa78b375dff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:02.588376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f3a09077-08be-4a50-a8ab-d9c8aeee55b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:02.591303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-665d8f54-8271-4456-a431-faa78b375dff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:02.596955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f3a09077-08be-4a50-a8ab-d9c8aeee55b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.228569 May 25 16:40:02.597572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 809/1594] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:02 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:02.600315 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-665d8f54-8271-4456-a431-faa78b375dff tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222201 May 25 16:40:02.600852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 786/1595] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:02 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:40:02.605607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:02.606196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:02.610875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ad7a9346-c01a-4882-be3a-28b69abbae8e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:02.611533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ad7a9346-c01a-4882-be3a-28b69abbae8e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:02.612137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:02.612591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:02.613347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:02.613778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:02.617433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad7a9346-c01a-4882-be3a-28b69abbae8e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:02.617883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad7a9346-c01a-4882-be3a-28b69abbae8e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:02.618651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad7a9346-c01a-4882-be3a-28b69abbae8e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:02.619062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad7a9346-c01a-4882-be3a-28b69abbae8e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:02.703482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ad7a9346-c01a-4882-be3a-28b69abbae8e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:02.711866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ad7a9346-c01a-4882-be3a-28b69abbae8e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.112678 May 25 16:40:02.712448 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 810/1596] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:02 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:02.767614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:02.776109 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:02.777615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:02.778814 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e98560a2-95b2-4432-afef-f163177a6658 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.176847 May 25 16:40:02.779390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 787/1597] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:02 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:03.615605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8e687880-57b0-440b-ab7c-8948dede4390 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:03.616216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8e687880-57b0-440b-ab7c-8948dede4390 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:03.617308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7d29506c-0a05-4d1f-ab65-18c47521196a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:03.617927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7d29506c-0a05-4d1f-ab65-18c47521196a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:03.623049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e687880-57b0-440b-ab7c-8948dede4390 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:03.623568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e687880-57b0-440b-ab7c-8948dede4390 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:03.624377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e687880-57b0-440b-ab7c-8948dede4390 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:03.624813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e687880-57b0-440b-ab7c-8948dede4390 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:03.624975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d29506c-0a05-4d1f-ab65-18c47521196a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:03.625372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d29506c-0a05-4d1f-ab65-18c47521196a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:03.626122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d29506c-0a05-4d1f-ab65-18c47521196a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:03.626548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7d29506c-0a05-4d1f-ab65-18c47521196a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:03.816587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7d29506c-0a05-4d1f-ab65-18c47521196a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:03.819513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8e687880-57b0-440b-ab7c-8948dede4390 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:03.825203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7d29506c-0a05-4d1f-ab65-18c47521196a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.211645 May 25 16:40:03.825772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 788/1598] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:03 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:03.827720 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8e687880-57b0-440b-ab7c-8948dede4390 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.216385 May 25 16:40:03.828472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 811/1599] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:03 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:03.830300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e3173d74-5b97-40aa-8744-ed63864bf3bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:03.830921 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e3173d74-5b97-40aa-8744-ed63864bf3bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:03.837120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3173d74-5b97-40aa-8744-ed63864bf3bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:03.837686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3173d74-5b97-40aa-8744-ed63864bf3bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:03.838442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3173d74-5b97-40aa-8744-ed63864bf3bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:03.838990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3173d74-5b97-40aa-8744-ed63864bf3bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:03.841035 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:03.841758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:03.847750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:03.848193 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:03.848941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:03.849370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:03.909941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e3173d74-5b97-40aa-8744-ed63864bf3bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:03.920022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e3173d74-5b97-40aa-8744-ed63864bf3bf tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092839 May 25 16:40:03.920717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 789/1600] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:03 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:03.992746 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:04.001413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:04.002920 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:04.004118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b37253d9-eae7-4a25-a94a-f6750bbcc8f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.166871 May 25 16:40:04.004684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 812/1601] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:03 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:04.848205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7e76a451-ad23-4267-82e8-862796267c99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:04.849257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7e76a451-ad23-4267-82e8-862796267c99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:04.849462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2523792f-05b4-45f7-bf30-dfd42a9b746c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:04.850535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2523792f-05b4-45f7-bf30-dfd42a9b746c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:04.860324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e76a451-ad23-4267-82e8-862796267c99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:04.861068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e76a451-ad23-4267-82e8-862796267c99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:04.861577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2523792f-05b4-45f7-bf30-dfd42a9b746c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:04.862309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2523792f-05b4-45f7-bf30-dfd42a9b746c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:04.862630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e76a451-ad23-4267-82e8-862796267c99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:04.863076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e76a451-ad23-4267-82e8-862796267c99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:04.863642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2523792f-05b4-45f7-bf30-dfd42a9b746c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:04.864363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2523792f-05b4-45f7-bf30-dfd42a9b746c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:05.059829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2523792f-05b4-45f7-bf30-dfd42a9b746c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:05.064127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7e76a451-ad23-4267-82e8-862796267c99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:05.068518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2523792f-05b4-45f7-bf30-dfd42a9b746c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.225656 May 25 16:40:05.069064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 813/1602] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:04 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:05.072554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7e76a451-ad23-4267-82e8-862796267c99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.230901 May 25 16:40:05.073159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 790/1603] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:04 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:05.074165 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2b7ea0df-8fa9-42b3-9a41-83fc1db96a4f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:05.074765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2b7ea0df-8fa9-42b3-9a41-83fc1db96a4f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:05.078201 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:05.079113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:05.080823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b7ea0df-8fa9-42b3-9a41-83fc1db96a4f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:05.081303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b7ea0df-8fa9-42b3-9a41-83fc1db96a4f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:05.082018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b7ea0df-8fa9-42b3-9a41-83fc1db96a4f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:05.082632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2b7ea0df-8fa9-42b3-9a41-83fc1db96a4f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:05.085624 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:05.086159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:05.087031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:05.087569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:05.170624 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2b7ea0df-8fa9-42b3-9a41-83fc1db96a4f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:05.178996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2b7ea0df-8fa9-42b3-9a41-83fc1db96a4f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.108570 May 25 16:40:05.179889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 814/1604] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:05 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:05.245622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:05.254344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:05.255884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:05.257047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8632bec1-3b07-4a7c-97e0-093c735c469f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.182810 May 25 16:40:05.257641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 791/1605] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:05 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:06.089749 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4a1b6b2a-bad1-42d8-90ce-4b081ae81739 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:06.090371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4a1b6b2a-bad1-42d8-90ce-4b081ae81739 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:06.091917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2ea616e7-01e6-4a6a-b9cf-b30653b1c0b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:06.092531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2ea616e7-01e6-4a6a-b9cf-b30653b1c0b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:06.097203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a1b6b2a-bad1-42d8-90ce-4b081ae81739 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:06.097636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a1b6b2a-bad1-42d8-90ce-4b081ae81739 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:06.098403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a1b6b2a-bad1-42d8-90ce-4b081ae81739 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:06.098840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a1b6b2a-bad1-42d8-90ce-4b081ae81739 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:06.099125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ea616e7-01e6-4a6a-b9cf-b30653b1c0b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:06.099612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ea616e7-01e6-4a6a-b9cf-b30653b1c0b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:06.100386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ea616e7-01e6-4a6a-b9cf-b30653b1c0b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:06.100820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ea616e7-01e6-4a6a-b9cf-b30653b1c0b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:06.290655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4a1b6b2a-bad1-42d8-90ce-4b081ae81739 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:06.294905 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2ea616e7-01e6-4a6a-b9cf-b30653b1c0b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:06.298858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4a1b6b2a-bad1-42d8-90ce-4b081ae81739 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.215366 May 25 16:40:06.299472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 815/1606] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:06 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:06.304081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dba1831d-f70d-4e41-ad05-5bf516596bb6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:06.304686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dba1831d-f70d-4e41-ad05-5bf516596bb6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:06.306180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2ea616e7-01e6-4a6a-b9cf-b30653b1c0b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.218637 May 25 16:40:06.307663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 792/1607] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:06 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:06.314945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:06.315598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:06.318572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dba1831d-f70d-4e41-ad05-5bf516596bb6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:06.319026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dba1831d-f70d-4e41-ad05-5bf516596bb6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:06.319817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dba1831d-f70d-4e41-ad05-5bf516596bb6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:06.320388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dba1831d-f70d-4e41-ad05-5bf516596bb6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:06.322076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:06.322533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:06.323368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:06.323864 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:06.398065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dba1831d-f70d-4e41-ad05-5bf516596bb6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:06.406539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dba1831d-f70d-4e41-ad05-5bf516596bb6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.106116 May 25 16:40:06.407138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 816/1608] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:06 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:06.484766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:06.497852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:06.500329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:06.502048 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e875cd1c-849e-4e6c-a4d1-139c7c5e69ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.192703 May 25 16:40:06.502764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 793/1609] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:06 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:07.320384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2949d3a1-8115-4196-9ba6-a09248d99b49 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:07.321387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2949d3a1-8115-4196-9ba6-a09248d99b49 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:07.326316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-80632f54-f3a1-462b-aad9-0585777070c3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:07.326863 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-80632f54-f3a1-462b-aad9-0585777070c3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:07.329366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2949d3a1-8115-4196-9ba6-a09248d99b49 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:07.329821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2949d3a1-8115-4196-9ba6-a09248d99b49 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:07.330588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2949d3a1-8115-4196-9ba6-a09248d99b49 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:07.331023 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2949d3a1-8115-4196-9ba6-a09248d99b49 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:07.334062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80632f54-f3a1-462b-aad9-0585777070c3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:07.334518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80632f54-f3a1-462b-aad9-0585777070c3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:07.335333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80632f54-f3a1-462b-aad9-0585777070c3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:07.335750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80632f54-f3a1-462b-aad9-0585777070c3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:07.528867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2949d3a1-8115-4196-9ba6-a09248d99b49 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:07.530770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-80632f54-f3a1-462b-aad9-0585777070c3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:07.537561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2949d3a1-8115-4196-9ba6-a09248d99b49 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.223688 May 25 16:40:07.538181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 817/1610] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:07 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:07.538897 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-80632f54-f3a1-462b-aad9-0585777070c3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217237 May 25 16:40:07.546036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 794/1611] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:07 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:07.546225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-31c9ccb6-bf2a-45bd-8795-4e6267175507 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:07.546225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-31c9ccb6-bf2a-45bd-8795-4e6267175507 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:07.546410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:07.546410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:07.550574 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:07.551027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:07.551728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31c9ccb6-bf2a-45bd-8795-4e6267175507 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:07.551853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:07.552183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31c9ccb6-bf2a-45bd-8795-4e6267175507 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:07.552301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:07.552940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31c9ccb6-bf2a-45bd-8795-4e6267175507 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:07.553374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31c9ccb6-bf2a-45bd-8795-4e6267175507 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:07.556208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:07.642970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-31c9ccb6-bf2a-45bd-8795-4e6267175507 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:07.651422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-31c9ccb6-bf2a-45bd-8795-4e6267175507 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.112131 May 25 16:40:07.652009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 818/1612] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:07 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:07.713164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:07.725364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:07.727741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:07.729537 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0adb34a4-61e6-4789-bb54-bdd3b046f144 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.188540 May 25 16:40:07.730352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 795/1613] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:07 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:08.558985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c4879b25-3d04-4a6e-af62-6c5cd9250509 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:08.559615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-912ee9bb-2e78-4afa-a2e7-402efdf75171 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:08.559979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c4879b25-3d04-4a6e-af62-6c5cd9250509 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:08.560074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-912ee9bb-2e78-4afa-a2e7-402efdf75171 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:08.567200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4879b25-3d04-4a6e-af62-6c5cd9250509 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:08.567659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4879b25-3d04-4a6e-af62-6c5cd9250509 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:08.568056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-912ee9bb-2e78-4afa-a2e7-402efdf75171 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:08.568418 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4879b25-3d04-4a6e-af62-6c5cd9250509 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:08.568524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-912ee9bb-2e78-4afa-a2e7-402efdf75171 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:08.568851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4879b25-3d04-4a6e-af62-6c5cd9250509 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:08.569251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-912ee9bb-2e78-4afa-a2e7-402efdf75171 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:08.569684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-912ee9bb-2e78-4afa-a2e7-402efdf75171 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:08.758751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c4879b25-3d04-4a6e-af62-6c5cd9250509 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:08.762884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-912ee9bb-2e78-4afa-a2e7-402efdf75171 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:08.767058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c4879b25-3d04-4a6e-af62-6c5cd9250509 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.213858 May 25 16:40:08.767672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 819/1614] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:08 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:08.771448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-912ee9bb-2e78-4afa-a2e7-402efdf75171 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217504 May 25 16:40:08.772198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 796/1615] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:08 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:08.772328 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b920814d-e23b-4e32-9262-5012e837591e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:08.772899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b920814d-e23b-4e32-9262-5012e837591e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:08.776679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:08.777308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:08.782815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b920814d-e23b-4e32-9262-5012e837591e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:08.783372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b920814d-e23b-4e32-9262-5012e837591e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:08.783484 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:08.783914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:08.784314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b920814d-e23b-4e32-9262-5012e837591e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:08.784677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:08.784788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b920814d-e23b-4e32-9262-5012e837591e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:08.785112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:08.853242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b920814d-e23b-4e32-9262-5012e837591e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:08.861548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b920814d-e23b-4e32-9262-5012e837591e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092891 May 25 16:40:08.862115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 820/1616] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:08 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:08.943047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:08.951907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:08.953421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:08.954579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bbf1d15d-8f67-4b55-b4a8-eadbddf24678 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.181660 May 25 16:40:08.955198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 797/1617] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:08 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:09.792456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-08e1cc3b-3636-4962-9c02-29fe0bec27b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:09.793426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d0007d90-f8c2-4e6b-a7ab-399281ef3ba4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:09.794042 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-08e1cc3b-3636-4962-9c02-29fe0bec27b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:09.794205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d0007d90-f8c2-4e6b-a7ab-399281ef3ba4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:09.804481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-08e1cc3b-3636-4962-9c02-29fe0bec27b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:09.804746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0007d90-f8c2-4e6b-a7ab-399281ef3ba4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:09.805222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-08e1cc3b-3636-4962-9c02-29fe0bec27b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:09.805490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0007d90-f8c2-4e6b-a7ab-399281ef3ba4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:09.806479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0007d90-f8c2-4e6b-a7ab-399281ef3ba4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:09.806724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-08e1cc3b-3636-4962-9c02-29fe0bec27b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:09.807217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-08e1cc3b-3636-4962-9c02-29fe0bec27b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:09.807526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0007d90-f8c2-4e6b-a7ab-399281ef3ba4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:09.998885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d0007d90-f8c2-4e6b-a7ab-399281ef3ba4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:10.002328 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-08e1cc3b-3636-4962-9c02-29fe0bec27b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:10.004661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:10.007226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d0007d90-f8c2-4e6b-a7ab-399281ef3ba4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221482 May 25 16:40:10.008026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 798/1618] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:09 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:10.011896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-08e1cc3b-3636-4962-9c02-29fe0bec27b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.226258 May 25 16:40:10.012217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0a6627b8-d8de-41f9-8e3c-75bde09b3ada tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:10.012645 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 821/1619] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:09 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:10.012820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0a6627b8-d8de-41f9-8e3c-75bde09b3ada tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:10.015561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=97 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:10.018999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a6627b8-d8de-41f9-8e3c-75bde09b3ada tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:10.019499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a6627b8-d8de-41f9-8e3c-75bde09b3ada tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:10.020271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a6627b8-d8de-41f9-8e3c-75bde09b3ada tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:10.020703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a6627b8-d8de-41f9-8e3c-75bde09b3ada tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:10.025958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:10.026669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:10.033086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:10.033505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:10.034397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:10.034837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:10.101457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0a6627b8-d8de-41f9-8e3c-75bde09b3ada tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:10.110078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0a6627b8-d8de-41f9-8e3c-75bde09b3ada tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.101270 May 25 16:40:10.110631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 799/1620] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:10 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:10.186523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:10.195370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:10.196867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:10.198141 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dedc33ad-5bcf-47dd-822f-03f9d9ab83ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.175745 May 25 16:40:10.198720 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 822/1621] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:10 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:11.030323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-18c160b3-45e7-40f4-907d-acb0bdc5bffc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:11.031393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-18c160b3-45e7-40f4-907d-acb0bdc5bffc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:11.032866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d690f422-d721-4fde-8eeb-ef680cebbedb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:11.033874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d690f422-d721-4fde-8eeb-ef680cebbedb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:11.042738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-18c160b3-45e7-40f4-907d-acb0bdc5bffc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:11.043196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-18c160b3-45e7-40f4-907d-acb0bdc5bffc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:11.043586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d690f422-d721-4fde-8eeb-ef680cebbedb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:11.043974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-18c160b3-45e7-40f4-907d-acb0bdc5bffc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:11.044115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d690f422-d721-4fde-8eeb-ef680cebbedb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:11.044390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-18c160b3-45e7-40f4-907d-acb0bdc5bffc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:11.044774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d690f422-d721-4fde-8eeb-ef680cebbedb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:11.045208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d690f422-d721-4fde-8eeb-ef680cebbedb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:11.235738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-18c160b3-45e7-40f4-907d-acb0bdc5bffc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:11.238581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d690f422-d721-4fde-8eeb-ef680cebbedb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:11.244070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-18c160b3-45e7-40f4-907d-acb0bdc5bffc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220447 May 25 16:40:11.244650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 800/1622] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:11 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:11.247438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d690f422-d721-4fde-8eeb-ef680cebbedb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.221121 May 25 16:40:11.248265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 823/1623] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:11 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:11.249380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-32bc6b47-8956-48a4-a4a3-6bc1713fdc69 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:11.249986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-32bc6b47-8956-48a4-a4a3-6bc1713fdc69 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:11.255896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32bc6b47-8956-48a4-a4a3-6bc1713fdc69 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:11.256353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32bc6b47-8956-48a4-a4a3-6bc1713fdc69 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:11.257114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32bc6b47-8956-48a4-a4a3-6bc1713fdc69 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:11.257536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-32bc6b47-8956-48a4-a4a3-6bc1713fdc69 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:11.259693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:11.260303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:11.266399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:11.266838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:11.267650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:11.268081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:11.321324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-32bc6b47-8956-48a4-a4a3-6bc1713fdc69 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:11.329505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-32bc6b47-8956-48a4-a4a3-6bc1713fdc69 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083885 May 25 16:40:11.330084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 801/1624] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:11 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:11.417548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:11.426130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:11.427667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:11.428836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-63a829a3-80e5-4688-a081-c96172432382 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.172844 May 25 16:40:11.429376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 824/1625] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:11 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:12.261360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-898fe599-4a61-45e1-a9bf-2044cf8c8512 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:12.261988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-898fe599-4a61-45e1-a9bf-2044cf8c8512 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:12.264860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ea5ead08-165e-4ae6-b3c3-42d88032d7f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:12.265464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ea5ead08-165e-4ae6-b3c3-42d88032d7f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:12.268768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-898fe599-4a61-45e1-a9bf-2044cf8c8512 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:12.269222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-898fe599-4a61-45e1-a9bf-2044cf8c8512 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:12.269988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-898fe599-4a61-45e1-a9bf-2044cf8c8512 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:12.270424 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-898fe599-4a61-45e1-a9bf-2044cf8c8512 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:12.272089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea5ead08-165e-4ae6-b3c3-42d88032d7f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:12.272503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea5ead08-165e-4ae6-b3c3-42d88032d7f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:12.273289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea5ead08-165e-4ae6-b3c3-42d88032d7f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:12.273768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea5ead08-165e-4ae6-b3c3-42d88032d7f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:12.462015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-898fe599-4a61-45e1-a9bf-2044cf8c8512 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:12.464919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ea5ead08-165e-4ae6-b3c3-42d88032d7f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:12.470578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-898fe599-4a61-45e1-a9bf-2044cf8c8512 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.213560 May 25 16:40:12.471209 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 802/1626] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:12 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:12.473368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ea5ead08-165e-4ae6-b3c3-42d88032d7f8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.212294 May 25 16:40:12.476040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-35b7ef93-6936-4a3b-8f39-9f61673fa35e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:12.476661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-35b7ef93-6936-4a3b-8f39-9f61673fa35e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:12.480336 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 825/1627] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:12 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:12.482793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35b7ef93-6936-4a3b-8f39-9f61673fa35e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:12.483296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35b7ef93-6936-4a3b-8f39-9f61673fa35e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:12.484060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35b7ef93-6936-4a3b-8f39-9f61673fa35e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:12.484490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35b7ef93-6936-4a3b-8f39-9f61673fa35e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:12.485066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:12.485663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:12.491531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:12.492089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:12.492840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:12.493269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:12.568141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-35b7ef93-6936-4a3b-8f39-9f61673fa35e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:12.579787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-35b7ef93-6936-4a3b-8f39-9f61673fa35e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.107348 May 25 16:40:12.580400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 803/1628] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:12 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:12.638764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:12.651951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:12.654064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:12.655871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1fb426f3-4dca-4517-91ae-f3b4ef60f37b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.174392 May 25 16:40:12.656795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 826/1629] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:12 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:13.492086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b0e19dbc-a41d-4c2e-a3d5-d0c84bc2205c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:13.492915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b0e19dbc-a41d-4c2e-a3d5-d0c84bc2205c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:13.494226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c5a583d6-cbde-4f95-91b8-c0ef80d758bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:13.495480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c5a583d6-cbde-4f95-91b8-c0ef80d758bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:13.504222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0e19dbc-a41d-4c2e-a3d5-d0c84bc2205c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:13.504959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0e19dbc-a41d-4c2e-a3d5-d0c84bc2205c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:13.506202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0e19dbc-a41d-4c2e-a3d5-d0c84bc2205c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:13.506450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5a583d6-cbde-4f95-91b8-c0ef80d758bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:13.507126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b0e19dbc-a41d-4c2e-a3d5-d0c84bc2205c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:13.507408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5a583d6-cbde-4f95-91b8-c0ef80d758bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:13.508442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5a583d6-cbde-4f95-91b8-c0ef80d758bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:13.509172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5a583d6-cbde-4f95-91b8-c0ef80d758bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:13.711316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b0e19dbc-a41d-4c2e-a3d5-d0c84bc2205c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:13.715673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c5a583d6-cbde-4f95-91b8-c0ef80d758bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:13.719763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b0e19dbc-a41d-4c2e-a3d5-d0c84bc2205c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.234322 May 25 16:40:13.720331 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 804/1630] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:13 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:13.723910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c5a583d6-cbde-4f95-91b8-c0ef80d758bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.236283 May 25 16:40:13.724643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 827/1631] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:13 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:40:13.724763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d0d70b01-d518-4ace-b0a6-476580cbd2c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:13.725368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d0d70b01-d518-4ace-b0a6-476580cbd2c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:13.731290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0d70b01-d518-4ace-b0a6-476580cbd2c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:13.731860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0d70b01-d518-4ace-b0a6-476580cbd2c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:13.732624 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0d70b01-d518-4ace-b0a6-476580cbd2c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:13.733049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d0d70b01-d518-4ace-b0a6-476580cbd2c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:13.736935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:13.737610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:13.743788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:13.744269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:13.745175 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:13.745678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:13.803604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d0d70b01-d518-4ace-b0a6-476580cbd2c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:13.812554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d0d70b01-d518-4ace-b0a6-476580cbd2c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091208 May 25 16:40:13.813122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 805/1632] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:13 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:13.897083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:13.909825 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:13.912275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:13.914018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4a3034db-8645-4b37-acfa-e63038d62bab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.180730 May 25 16:40:13.915124 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 828/1633] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:13 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:14.741338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fec518e8-a2f3-4bca-8d5f-42567df9a5b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:14.742382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fec518e8-a2f3-4bca-8d5f-42567df9a5b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:14.744856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-339ef393-5781-4014-965b-ce068ad247bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:14.746055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-339ef393-5781-4014-965b-ce068ad247bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:14.753721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec518e8-a2f3-4bca-8d5f-42567df9a5b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:14.754404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec518e8-a2f3-4bca-8d5f-42567df9a5b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:14.755764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec518e8-a2f3-4bca-8d5f-42567df9a5b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:14.756488 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec518e8-a2f3-4bca-8d5f-42567df9a5b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:14.756730 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-339ef393-5781-4014-965b-ce068ad247bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:14.757410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-339ef393-5781-4014-965b-ce068ad247bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:14.758652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-339ef393-5781-4014-965b-ce068ad247bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:14.759444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-339ef393-5781-4014-965b-ce068ad247bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:14.949893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fec518e8-a2f3-4bca-8d5f-42567df9a5b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:14.952829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-339ef393-5781-4014-965b-ce068ad247bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:14.958163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fec518e8-a2f3-4bca-8d5f-42567df9a5b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.223385 May 25 16:40:14.958739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 806/1634] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:14 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:14.960849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-339ef393-5781-4014-965b-ce068ad247bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.222532 May 25 16:40:14.961562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 829/1635] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:14 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:14.963196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-215f9e28-aa17-4671-b78a-2d34ffee9f13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:14.963798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-215f9e28-aa17-4671-b78a-2d34ffee9f13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:14.970859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:14.971421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-215f9e28-aa17-4671-b78a-2d34ffee9f13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:14.971563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:14.971886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-215f9e28-aa17-4671-b78a-2d34ffee9f13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:14.972647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-215f9e28-aa17-4671-b78a-2d34ffee9f13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:14.973077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-215f9e28-aa17-4671-b78a-2d34ffee9f13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:14.977507 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:14.978047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:14.978866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:14.979432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:15.056011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-215f9e28-aa17-4671-b78a-2d34ffee9f13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:15.064515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-215f9e28-aa17-4671-b78a-2d34ffee9f13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.104676 May 25 16:40:15.064984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 807/1636] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:14 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:15.127284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:15.140758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:15.143099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:15.144845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-104b395c-0a61-4db5-a76b-6f5239e4ffa8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.177407 May 25 16:40:15.145496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 830/1637] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:14 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:15.980121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4f91f43e-880d-4d9d-bac9-291c49802f7b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:15.981051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-685a9543-ed31-47ab-8885-33a05c80f6b6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:15.981661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4f91f43e-880d-4d9d-bac9-291c49802f7b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:15.981945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-685a9543-ed31-47ab-8885-33a05c80f6b6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:15.992436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-685a9543-ed31-47ab-8885-33a05c80f6b6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:15.993108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f91f43e-880d-4d9d-bac9-291c49802f7b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:15.993361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-685a9543-ed31-47ab-8885-33a05c80f6b6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:15.993873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f91f43e-880d-4d9d-bac9-291c49802f7b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:15.994410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-685a9543-ed31-47ab-8885-33a05c80f6b6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:15.995089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-685a9543-ed31-47ab-8885-33a05c80f6b6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:15.995258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f91f43e-880d-4d9d-bac9-291c49802f7b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:15.995676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f91f43e-880d-4d9d-bac9-291c49802f7b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:16.200535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-685a9543-ed31-47ab-8885-33a05c80f6b6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:16.203504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4f91f43e-880d-4d9d-bac9-291c49802f7b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:16.209079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-685a9543-ed31-47ab-8885-33a05c80f6b6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.234507 May 25 16:40:16.209886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 831/1638] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:15 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:16.212102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4f91f43e-880d-4d9d-bac9-291c49802f7b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.238573 May 25 16:40:16.219699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 808/1639] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:15 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:16.219910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-da1e22b9-b07e-4a27-a1f3-14ce7f093ac0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:16.219910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-da1e22b9-b07e-4a27-a1f3-14ce7f093ac0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:16.220099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:16.220099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:16.221316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da1e22b9-b07e-4a27-a1f3-14ce7f093ac0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:16.221774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da1e22b9-b07e-4a27-a1f3-14ce7f093ac0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:16.222780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da1e22b9-b07e-4a27-a1f3-14ce7f093ac0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:16.223172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-da1e22b9-b07e-4a27-a1f3-14ce7f093ac0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:16.224257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:16.224744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:16.225575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:16.226033 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:16.295773 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-da1e22b9-b07e-4a27-a1f3-14ce7f093ac0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:16.304216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-da1e22b9-b07e-4a27-a1f3-14ce7f093ac0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093580 May 25 16:40:16.304795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 832/1640] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:16 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:16.378598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:16.387430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:16.388925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:16.390083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bcdf2126-c901-4f00-8eea-c8d2a8d68a88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.176387 May 25 16:40:16.390643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 809/1641] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:16 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:17.230287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9976529d-18b3-439c-aacf-7ab87999b655 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:17.230870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9976529d-18b3-439c-aacf-7ab87999b655 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:17.232223 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-775c01a5-ebbb-4dab-86c8-ba32e779192d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:17.232827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-775c01a5-ebbb-4dab-86c8-ba32e779192d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:17.233238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=98 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:17.239488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9976529d-18b3-439c-aacf-7ab87999b655 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:17.239800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-775c01a5-ebbb-4dab-86c8-ba32e779192d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:17.239928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9976529d-18b3-439c-aacf-7ab87999b655 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:17.240270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-775c01a5-ebbb-4dab-86c8-ba32e779192d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:17.240700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9976529d-18b3-439c-aacf-7ab87999b655 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:17.240999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-775c01a5-ebbb-4dab-86c8-ba32e779192d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:17.241106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9976529d-18b3-439c-aacf-7ab87999b655 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:17.241421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-775c01a5-ebbb-4dab-86c8-ba32e779192d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:17.436835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-775c01a5-ebbb-4dab-86c8-ba32e779192d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:17.441317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9976529d-18b3-439c-aacf-7ab87999b655 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:17.448660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-775c01a5-ebbb-4dab-86c8-ba32e779192d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221319 May 25 16:40:17.449824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 810/1642] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:17 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:17.452720 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9976529d-18b3-439c-aacf-7ab87999b655 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.228589 May 25 16:40:17.454655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 833/1643] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:17 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:17.454805 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e4e9fc0c-7e3b-4264-a96b-9d5ac9d0d7e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:17.454805 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e4e9fc0c-7e3b-4264-a96b-9d5ac9d0d7e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:17.459738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:17.460671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:17.460985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e4e9fc0c-7e3b-4264-a96b-9d5ac9d0d7e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:17.461669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e4e9fc0c-7e3b-4264-a96b-9d5ac9d0d7e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:17.462796 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e4e9fc0c-7e3b-4264-a96b-9d5ac9d0d7e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:17.463425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e4e9fc0c-7e3b-4264-a96b-9d5ac9d0d7e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:17.467819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:17.468306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:17.469126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:17.469565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:17.545886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e4e9fc0c-7e3b-4264-a96b-9d5ac9d0d7e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:17.548082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:17.555278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e4e9fc0c-7e3b-4264-a96b-9d5ac9d0d7e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.104704 May 25 16:40:17.555914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 811/1644] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:17 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:17.613287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:17.623289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:17.625103 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:17.626359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ca921dee-dbdf-4afe-9c19-439759da53c9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.172001 May 25 16:40:17.627402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 834/1645] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:17 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:18.466263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7e3eff69-d58c-44a1-bd74-3d19e285eccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:18.466872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7e3eff69-d58c-44a1-bd74-3d19e285eccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:18.470985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8caab91f-35f9-41c2-906d-a74fd7991002 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:18.471619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8caab91f-35f9-41c2-906d-a74fd7991002 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:18.473832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e3eff69-d58c-44a1-bd74-3d19e285eccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:18.474288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e3eff69-d58c-44a1-bd74-3d19e285eccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:18.475059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e3eff69-d58c-44a1-bd74-3d19e285eccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:18.475534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7e3eff69-d58c-44a1-bd74-3d19e285eccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:18.477831 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8caab91f-35f9-41c2-906d-a74fd7991002 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:18.478303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8caab91f-35f9-41c2-906d-a74fd7991002 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:18.479053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8caab91f-35f9-41c2-906d-a74fd7991002 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:18.479585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8caab91f-35f9-41c2-906d-a74fd7991002 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:18.680650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7e3eff69-d58c-44a1-bd74-3d19e285eccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:18.684264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8caab91f-35f9-41c2-906d-a74fd7991002 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:18.688889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7e3eff69-d58c-44a1-bd74-3d19e285eccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.226583 May 25 16:40:18.689467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 812/1646] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:18 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:18.692716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8caab91f-35f9-41c2-906d-a74fd7991002 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.225564 May 25 16:40:18.693463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 835/1647] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:18 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:18.694225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2d267950-f372-42e8-ba7a-5d1f8b2c8295 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:18.694824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2d267950-f372-42e8-ba7a-5d1f8b2c8295 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:18.700878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d267950-f372-42e8-ba7a-5d1f8b2c8295 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:18.701342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d267950-f372-42e8-ba7a-5d1f8b2c8295 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:18.702118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d267950-f372-42e8-ba7a-5d1f8b2c8295 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:18.702573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d267950-f372-42e8-ba7a-5d1f8b2c8295 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:18.706941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:18.707568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:18.713485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:18.713996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:18.714735 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:18.715186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:18.776398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2d267950-f372-42e8-ba7a-5d1f8b2c8295 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:18.784849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2d267950-f372-42e8-ba7a-5d1f8b2c8295 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.094425 May 25 16:40:18.785425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 813/1648] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:18 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:18.874812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:18.887482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:18.889436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:18.891219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c3d5f975-b4d4-4e18-be75-a5a2ec1d138b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.187621 May 25 16:40:18.892014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 836/1649] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:19.707107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dd930b24-8e7c-4c9f-9b01-95eea6a0ebee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:19.707842 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dd930b24-8e7c-4c9f-9b01-95eea6a0ebee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:19.710864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4eb0aa1c-25bf-4d77-a1df-393e4110c3dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:19.711616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4eb0aa1c-25bf-4d77-a1df-393e4110c3dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:19.715623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd930b24-8e7c-4c9f-9b01-95eea6a0ebee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:19.716073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd930b24-8e7c-4c9f-9b01-95eea6a0ebee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:19.716830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd930b24-8e7c-4c9f-9b01-95eea6a0ebee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:19.717257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd930b24-8e7c-4c9f-9b01-95eea6a0ebee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:19.718633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4eb0aa1c-25bf-4d77-a1df-393e4110c3dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:19.719090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4eb0aa1c-25bf-4d77-a1df-393e4110c3dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:19.719887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4eb0aa1c-25bf-4d77-a1df-393e4110c3dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:19.720323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4eb0aa1c-25bf-4d77-a1df-393e4110c3dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:19.924683 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dd930b24-8e7c-4c9f-9b01-95eea6a0ebee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:19.929280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4eb0aa1c-25bf-4d77-a1df-393e4110c3dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:19.937361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dd930b24-8e7c-4c9f-9b01-95eea6a0ebee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.234437 May 25 16:40:19.938289 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 814/1650] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:19 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:19.941847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4eb0aa1c-25bf-4d77-a1df-393e4110c3dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.235186 May 25 16:40:19.942617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 837/1651] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:19 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:19.944390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5e9ada9d-343f-44bf-980b-1a3736d4783f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:19.945213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5e9ada9d-343f-44bf-980b-1a3736d4783f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:19.948302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:19.948859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:19.952018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e9ada9d-343f-44bf-980b-1a3736d4783f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:19.952556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e9ada9d-343f-44bf-980b-1a3736d4783f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:19.953360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e9ada9d-343f-44bf-980b-1a3736d4783f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:19.953863 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5e9ada9d-343f-44bf-980b-1a3736d4783f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:19.955081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:19.955646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:19.956458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:19.956903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:19.973840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:20.022211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5e9ada9d-343f-44bf-980b-1a3736d4783f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:20.030397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5e9ada9d-343f-44bf-980b-1a3736d4783f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090853 May 25 16:40:20.030952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 815/1652] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:19 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:20.106518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:20.115255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:20.116783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:20.117990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ad3c5641-547e-4716-8130-91ec4ceaecdf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.174184 May 25 16:40:20.118558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 838/1653] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:19 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:20.957933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bb4422f2-f3d1-42c5-bc9d-0441385b478d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:20.958553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bb4422f2-f3d1-42c5-bc9d-0441385b478d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:20.959471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e5e1993c-62d7-4fc3-aba8-7d2ae47a83c4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:20.960348 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e5e1993c-62d7-4fc3-aba8-7d2ae47a83c4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:20.965681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bb4422f2-f3d1-42c5-bc9d-0441385b478d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:20.966288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bb4422f2-f3d1-42c5-bc9d-0441385b478d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:20.967047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5e1993c-62d7-4fc3-aba8-7d2ae47a83c4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:20.967197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bb4422f2-f3d1-42c5-bc9d-0441385b478d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:20.967557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bb4422f2-f3d1-42c5-bc9d-0441385b478d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:20.967694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5e1993c-62d7-4fc3-aba8-7d2ae47a83c4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:20.968353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5e1993c-62d7-4fc3-aba8-7d2ae47a83c4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:20.968819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5e1993c-62d7-4fc3-aba8-7d2ae47a83c4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:21.159440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bb4422f2-f3d1-42c5-bc9d-0441385b478d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:21.162915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e5e1993c-62d7-4fc3-aba8-7d2ae47a83c4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:21.168084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bb4422f2-f3d1-42c5-bc9d-0441385b478d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.214340 May 25 16:40:21.168697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 816/1654] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:20 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:21.172099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e5e1993c-62d7-4fc3-aba8-7d2ae47a83c4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.216708 May 25 16:40:21.172859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 839/1655] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:20 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:40:21.173289 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f2f8e37c-b62d-4440-942a-5a64741a4bd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:21.173878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f2f8e37c-b62d-4440-942a-5a64741a4bd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:21.180285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:21.180285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:21.180817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2f8e37c-b62d-4440-942a-5a64741a4bd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:21.181274 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2f8e37c-b62d-4440-942a-5a64741a4bd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:21.182044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2f8e37c-b62d-4440-942a-5a64741a4bd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:21.182738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2f8e37c-b62d-4440-942a-5a64741a4bd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:21.184414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:21.184884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:21.185632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:21.186068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:21.264799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f2f8e37c-b62d-4440-942a-5a64741a4bd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:21.273032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f2f8e37c-b62d-4440-942a-5a64741a4bd8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.103384 May 25 16:40:21.273674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 817/1656] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:21 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:21.333490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:21.343009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:21.344659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:21.345839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-56bcdd40-6bfd-42e6-88c2-4a8c23700537 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.172106 May 25 16:40:21.346403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 840/1657] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:21 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:22.192456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d5d3e861-9ca1-44e8-b195-258e6df0a36c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:22.193465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d5d3e861-9ca1-44e8-b195-258e6df0a36c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:22.194010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-27e8b3dd-5037-4a8f-b6fc-e63f992db3b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:22.195118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-27e8b3dd-5037-4a8f-b6fc-e63f992db3b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:22.202413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5d3e861-9ca1-44e8-b195-258e6df0a36c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:22.202830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27e8b3dd-5037-4a8f-b6fc-e63f992db3b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:22.202998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5d3e861-9ca1-44e8-b195-258e6df0a36c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:22.203321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27e8b3dd-5037-4a8f-b6fc-e63f992db3b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:22.203652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5d3e861-9ca1-44e8-b195-258e6df0a36c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:22.204082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27e8b3dd-5037-4a8f-b6fc-e63f992db3b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:22.204245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5d3e861-9ca1-44e8-b195-258e6df0a36c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:22.204519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-27e8b3dd-5037-4a8f-b6fc-e63f992db3b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:22.397344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d5d3e861-9ca1-44e8-b195-258e6df0a36c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:22.400524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-27e8b3dd-5037-4a8f-b6fc-e63f992db3b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:22.405388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d5d3e861-9ca1-44e8-b195-258e6df0a36c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220330 May 25 16:40:22.406160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 818/1658] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:22 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:22.410508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-65960ecc-d2eb-4b59-a65e-0c663d5b3227 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:22.411107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-65960ecc-d2eb-4b59-a65e-0c663d5b3227 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:22.414623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-27e8b3dd-5037-4a8f-b6fc-e63f992db3b7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.227619 May 25 16:40:22.424459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 841/1659] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:22 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:22.424681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-65960ecc-d2eb-4b59-a65e-0c663d5b3227 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:22.424681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-65960ecc-d2eb-4b59-a65e-0c663d5b3227 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:22.424681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-65960ecc-d2eb-4b59-a65e-0c663d5b3227 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:22.424681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-65960ecc-d2eb-4b59-a65e-0c663d5b3227 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:22.425035 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:22.425035 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:22.426450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:22.426901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:22.427702 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:22.428131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:22.506278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-65960ecc-d2eb-4b59-a65e-0c663d5b3227 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:22.514758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-65960ecc-d2eb-4b59-a65e-0c663d5b3227 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.107796 May 25 16:40:22.515371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 819/1660] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:22 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:22.571998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:22.584477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:22.586592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:22.588322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-547592e6-4aa5-418c-81f8-39a28f9f8db6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.171901 May 25 16:40:22.589042 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 842/1661] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:22 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:23.432905 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bb215303-6b1e-4924-abfe-314b44e2b5d6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:23.433462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bb215303-6b1e-4924-abfe-314b44e2b5d6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:23.435076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a8ca92bf-6f93-4bfc-84b8-2fc26f162610 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:23.435718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a8ca92bf-6f93-4bfc-84b8-2fc26f162610 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:23.440448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bb215303-6b1e-4924-abfe-314b44e2b5d6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:23.440896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bb215303-6b1e-4924-abfe-314b44e2b5d6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:23.441656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bb215303-6b1e-4924-abfe-314b44e2b5d6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:23.442078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bb215303-6b1e-4924-abfe-314b44e2b5d6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:23.442378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8ca92bf-6f93-4bfc-84b8-2fc26f162610 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:23.442828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8ca92bf-6f93-4bfc-84b8-2fc26f162610 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:23.443638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8ca92bf-6f93-4bfc-84b8-2fc26f162610 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:23.444075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8ca92bf-6f93-4bfc-84b8-2fc26f162610 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:23.871123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bb215303-6b1e-4924-abfe-314b44e2b5d6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:23.874043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a8ca92bf-6f93-4bfc-84b8-2fc26f162610 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:23.879384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bb215303-6b1e-4924-abfe-314b44e2b5d6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.454872 May 25 16:40:23.879957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 820/1662] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:23 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:23.882574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a8ca92bf-6f93-4bfc-84b8-2fc26f162610 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.452385 May 25 16:40:23.883327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 843/1663] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:23 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:23.884337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8f9fb70f-6595-4692-aee7-8d5cf4c30010 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:23.884936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8f9fb70f-6595-4692-aee7-8d5cf4c30010 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:23.892132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:23.892740 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:23.894345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8f9fb70f-6595-4692-aee7-8d5cf4c30010 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:23.894799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8f9fb70f-6595-4692-aee7-8d5cf4c30010 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:23.895644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8f9fb70f-6595-4692-aee7-8d5cf4c30010 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:23.896135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8f9fb70f-6595-4692-aee7-8d5cf4c30010 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:23.898888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:23.899380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:23.900145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:23.900585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:23.979055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8f9fb70f-6595-4692-aee7-8d5cf4c30010 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:23.987524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8f9fb70f-6595-4692-aee7-8d5cf4c30010 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.106582 May 25 16:40:23.988134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 821/1664] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:23 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:24.050913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:24.060006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:24.061503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:24.062756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-77a68e15-cc17-463c-9b0b-5c45281d019f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.174233 May 25 16:40:24.063314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 844/1665] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:23 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:24.900810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c4bb42d1-028b-4485-984d-38e252f970ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:24.901624 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c4bb42d1-028b-4485-984d-38e252f970ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:24.902738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fbfe0a61-bf60-4cc5-9915-6d7ef9a81dcc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:24.903365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fbfe0a61-bf60-4cc5-9915-6d7ef9a81dcc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:24.908538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4bb42d1-028b-4485-984d-38e252f970ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:24.908993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4bb42d1-028b-4485-984d-38e252f970ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:24.909756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4bb42d1-028b-4485-984d-38e252f970ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:24.910028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fbfe0a61-bf60-4cc5-9915-6d7ef9a81dcc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:24.910265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4bb42d1-028b-4485-984d-38e252f970ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:24.910486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fbfe0a61-bf60-4cc5-9915-6d7ef9a81dcc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:24.911270 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fbfe0a61-bf60-4cc5-9915-6d7ef9a81dcc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:24.911703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fbfe0a61-bf60-4cc5-9915-6d7ef9a81dcc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:25.099982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c4bb42d1-028b-4485-984d-38e252f970ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:25.103189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fbfe0a61-bf60-4cc5-9915-6d7ef9a81dcc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:25.108788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c4bb42d1-028b-4485-984d-38e252f970ee tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.214437 May 25 16:40:25.109344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 822/1666] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:24 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:25.111734 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fbfe0a61-bf60-4cc5-9915-6d7ef9a81dcc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.214542 May 25 16:40:25.112539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 845/1667] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:24 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:25.113759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ca926e98-02d6-4202-97cb-1c4d91e73a6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:25.114364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ca926e98-02d6-4202-97cb-1c4d91e73a6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:25.120635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca926e98-02d6-4202-97cb-1c4d91e73a6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:25.121146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca926e98-02d6-4202-97cb-1c4d91e73a6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:25.122053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca926e98-02d6-4202-97cb-1c4d91e73a6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:25.122550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca926e98-02d6-4202-97cb-1c4d91e73a6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:25.123784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:25.124401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:25.131142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:25.131865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:25.132802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:25.133274 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:25.197072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ca926e98-02d6-4202-97cb-1c4d91e73a6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:25.205195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ca926e98-02d6-4202-97cb-1c4d91e73a6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.094877 May 25 16:40:25.205991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 823/1668] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:25 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:25.277996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:25.287065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:25.288620 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:25.290121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1e8f704b-a614-417d-9d6c-e1321bd2412d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.170448 May 25 16:40:25.290706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 846/1669] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:25 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:26.126763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-acc635fb-264b-42d6-af18-13c9cc453f5e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:26.127397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-acc635fb-264b-42d6-af18-13c9cc453f5e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:26.129268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-63bc4209-c4f5-4e04-9335-36a36515751a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:26.129886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-63bc4209-c4f5-4e04-9335-36a36515751a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:26.134471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-acc635fb-264b-42d6-af18-13c9cc453f5e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:26.134931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-acc635fb-264b-42d6-af18-13c9cc453f5e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:26.135847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-acc635fb-264b-42d6-af18-13c9cc453f5e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:26.136282 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-acc635fb-264b-42d6-af18-13c9cc453f5e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:26.136840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63bc4209-c4f5-4e04-9335-36a36515751a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:26.137285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63bc4209-c4f5-4e04-9335-36a36515751a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:26.138037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63bc4209-c4f5-4e04-9335-36a36515751a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:26.138464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63bc4209-c4f5-4e04-9335-36a36515751a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:26.339737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-acc635fb-264b-42d6-af18-13c9cc453f5e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:26.343333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-63bc4209-c4f5-4e04-9335-36a36515751a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:26.348101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-acc635fb-264b-42d6-af18-13c9cc453f5e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.225471 May 25 16:40:26.348667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 824/1670] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:26 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:26.351808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-63bc4209-c4f5-4e04-9335-36a36515751a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.226484 May 25 16:40:26.352536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 847/1671] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:26 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:26.353066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-40d19641-c9fe-4e1f-880c-0a1fe673ba97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:26.353665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-40d19641-c9fe-4e1f-880c-0a1fe673ba97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:26.357016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:26.357620 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:26.359616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-40d19641-c9fe-4e1f-880c-0a1fe673ba97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:26.360066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-40d19641-c9fe-4e1f-880c-0a1fe673ba97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:26.360819 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-40d19641-c9fe-4e1f-880c-0a1fe673ba97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:26.361249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-40d19641-c9fe-4e1f-880c-0a1fe673ba97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:26.363974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:26.364431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:26.365228 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:26.365666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:26.445403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-40d19641-c9fe-4e1f-880c-0a1fe673ba97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:26.456145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-40d19641-c9fe-4e1f-880c-0a1fe673ba97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.106182 May 25 16:40:26.456911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 825/1672] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:26.518541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:26.527459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:26.529095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:26.530247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bce5ab5c-a740-47f0-8ad9-00f97f114b02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.176903 May 25 16:40:26.530802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 848/1673] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:27.374460 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ef854916-9122-461d-b93e-a5d778505efa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:27.377490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ef854916-9122-461d-b93e-a5d778505efa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:27.386281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef854916-9122-461d-b93e-a5d778505efa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:27.387044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef854916-9122-461d-b93e-a5d778505efa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:27.388544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef854916-9122-461d-b93e-a5d778505efa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:27.388946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef854916-9122-461d-b93e-a5d778505efa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:27.549346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ef854916-9122-461d-b93e-a5d778505efa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:27.558692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ef854916-9122-461d-b93e-a5d778505efa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.190668 May 25 16:40:27.559280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 826/1674] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:27 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:27.563842 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e4fb7f35-36c3-44aa-9978-ece569e78c6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:27.564466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e4fb7f35-36c3-44aa-9978-ece569e78c6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:27.570463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e4fb7f35-36c3-44aa-9978-ece569e78c6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:27.571123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e4fb7f35-36c3-44aa-9978-ece569e78c6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:27.571928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e4fb7f35-36c3-44aa-9978-ece569e78c6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:27.572354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e4fb7f35-36c3-44aa-9978-ece569e78c6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:27.639878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e4fb7f35-36c3-44aa-9978-ece569e78c6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:27.651297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e4fb7f35-36c3-44aa-9978-ece569e78c6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090983 May 25 16:40:27.652008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 827/1675] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:27 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:27.657582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:27.658497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:27.665003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:27.665438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:27.666199 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:27.666798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:27.681934 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:27.734307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a589c76d-723b-46ca-a84d-3ab7b1f54887 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:27.735924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a589c76d-723b-46ca-a84d-3ab7b1f54887 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:27.742898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a589c76d-723b-46ca-a84d-3ab7b1f54887 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:27.743381 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a589c76d-723b-46ca-a84d-3ab7b1f54887 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:27.744345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a589c76d-723b-46ca-a84d-3ab7b1f54887 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:27.744971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a589c76d-723b-46ca-a84d-3ab7b1f54887 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:27.850540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:27.859553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:27.861062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:27.862222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-831cb69e-6aaa-42c8-96ab-60fd92ace49c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.209267 May 25 16:40:27.862775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 828/1676] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:27.893620 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a589c76d-723b-46ca-a84d-3ab7b1f54887 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:27.901943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a589c76d-723b-46ca-a84d-3ab7b1f54887 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.530488 May 25 16:40:27.902522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 849/1677] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:27 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:28.580361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cfcd3310-a120-49d3-9db6-cff6cece50bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:28.581438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cfcd3310-a120-49d3-9db6-cff6cece50bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:28.590702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfcd3310-a120-49d3-9db6-cff6cece50bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:28.591206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfcd3310-a120-49d3-9db6-cff6cece50bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:28.591961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfcd3310-a120-49d3-9db6-cff6cece50bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:28.592387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cfcd3310-a120-49d3-9db6-cff6cece50bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:28.668416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dc45bdaf-5090-41dc-85e4-3b5f5e967fe0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:28.669136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dc45bdaf-5090-41dc-85e4-3b5f5e967fe0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:28.676039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dc45bdaf-5090-41dc-85e4-3b5f5e967fe0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:28.676593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dc45bdaf-5090-41dc-85e4-3b5f5e967fe0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:28.677416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dc45bdaf-5090-41dc-85e4-3b5f5e967fe0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:28.677924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dc45bdaf-5090-41dc-85e4-3b5f5e967fe0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:28.756511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dc45bdaf-5090-41dc-85e4-3b5f5e967fe0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:28.760270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cfcd3310-a120-49d3-9db6-cff6cece50bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:28.767588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dc45bdaf-5090-41dc-85e4-3b5f5e967fe0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.103380 May 25 16:40:28.768169 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 850/1678] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:28.768425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cfcd3310-a120-49d3-9db6-cff6cece50bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.194938 May 25 16:40:28.768979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 829/1679] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:28 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:28.881039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:28.881591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:28.888133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:28.888943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:28.890434 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:28.890878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:28.919669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0998311a-06c3-4b50-b295-00a05035a160 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:28.920270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0998311a-06c3-4b50-b295-00a05035a160 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:28.926362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0998311a-06c3-4b50-b295-00a05035a160 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:28.926804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0998311a-06c3-4b50-b295-00a05035a160 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:28.927576 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0998311a-06c3-4b50-b295-00a05035a160 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:28.928001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0998311a-06c3-4b50-b295-00a05035a160 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:29.073381 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:29.084144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:29.085629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:29.086791 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d6dc1922-af18-4597-b476-c1e2ec9c51b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.209935 May 25 16:40:29.087384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 851/1680] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:29.101198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0998311a-06c3-4b50-b295-00a05035a160 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:29.109709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0998311a-06c3-4b50-b295-00a05035a160 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.193853 May 25 16:40:29.110272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 830/1681] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:28 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:29.784905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-908c63d6-6dc7-4d96-a4b3-e7b03a78f76d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:29.785489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-908c63d6-6dc7-4d96-a4b3-e7b03a78f76d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:29.786585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-34a604e3-5cba-4de5-890b-e22ffbb46165 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:29.787671 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-34a604e3-5cba-4de5-890b-e22ffbb46165 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:29.792655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-908c63d6-6dc7-4d96-a4b3-e7b03a78f76d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:29.793104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-908c63d6-6dc7-4d96-a4b3-e7b03a78f76d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:29.793856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-908c63d6-6dc7-4d96-a4b3-e7b03a78f76d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:29.794288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-908c63d6-6dc7-4d96-a4b3-e7b03a78f76d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:29.798627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34a604e3-5cba-4de5-890b-e22ffbb46165 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:29.799262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34a604e3-5cba-4de5-890b-e22ffbb46165 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:29.800367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34a604e3-5cba-4de5-890b-e22ffbb46165 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:29.800955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34a604e3-5cba-4de5-890b-e22ffbb46165 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:29.862907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-908c63d6-6dc7-4d96-a4b3-e7b03a78f76d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:29.871539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-908c63d6-6dc7-4d96-a4b3-e7b03a78f76d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090460 May 25 16:40:29.872467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 852/1682] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:29 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:29.955334 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-34a604e3-5cba-4de5-890b-e22ffbb46165 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:29.963677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-34a604e3-5cba-4de5-890b-e22ffbb46165 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.181626 May 25 16:40:29.964253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 831/1683] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:29 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:30.105513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:30.106112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:30.112871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:30.113257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:30.114004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:30.114431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:30.126929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e1926e58-f325-46cc-9aaa-d519eb394a85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:30.127561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e1926e58-f325-46cc-9aaa-d519eb394a85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:30.134179 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e1926e58-f325-46cc-9aaa-d519eb394a85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:30.134711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e1926e58-f325-46cc-9aaa-d519eb394a85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:30.135555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e1926e58-f325-46cc-9aaa-d519eb394a85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:30.136078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e1926e58-f325-46cc-9aaa-d519eb394a85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:30.310135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:30.314883 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e1926e58-f325-46cc-9aaa-d519eb394a85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:30.318809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:30.320327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:30.321484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a5218608-021e-4bdc-a07f-2b50dfaf7fc6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.220061 May 25 16:40:30.322033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 853/1684] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:30 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:30.323609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e1926e58-f325-46cc-9aaa-d519eb394a85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.200475 May 25 16:40:30.324166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 832/1685] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:30 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:30.891361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1d23d02f-e197-4d1e-9a63-746b66c75ffb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:30.892219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1d23d02f-e197-4d1e-9a63-746b66c75ffb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:30.898980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d23d02f-e197-4d1e-9a63-746b66c75ffb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:30.899461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d23d02f-e197-4d1e-9a63-746b66c75ffb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:30.900224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d23d02f-e197-4d1e-9a63-746b66c75ffb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:30.900651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d23d02f-e197-4d1e-9a63-746b66c75ffb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:30.974630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d23d02f-e197-4d1e-9a63-746b66c75ffb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:30.982297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8fe902c5-fac6-4b0e-955a-d5c1564371ab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:30.983181 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8fe902c5-fac6-4b0e-955a-d5c1564371ab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:30.983634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1d23d02f-e197-4d1e-9a63-746b66c75ffb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.098533 May 25 16:40:30.984230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 854/1686] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:30 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:30.990196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8fe902c5-fac6-4b0e-955a-d5c1564371ab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:30.990637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8fe902c5-fac6-4b0e-955a-d5c1564371ab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:30.991412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8fe902c5-fac6-4b0e-955a-d5c1564371ab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:30.991898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8fe902c5-fac6-4b0e-955a-d5c1564371ab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:31.153987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8fe902c5-fac6-4b0e-955a-d5c1564371ab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:31.162758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8fe902c5-fac6-4b0e-955a-d5c1564371ab tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.185318 May 25 16:40:31.163486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 833/1687] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:30 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:31.342114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-378a846e-2f59-42f3-ab17-c0556e7ee4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:31.342589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-378a846e-2f59-42f3-ab17-c0556e7ee4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:31.342766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:31.343449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:31.352422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-378a846e-2f59-42f3-ab17-c0556e7ee4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:31.352689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:31.353115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:31.353259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-378a846e-2f59-42f3-ab17-c0556e7ee4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:31.353881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:31.354318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:31.354520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-378a846e-2f59-42f3-ab17-c0556e7ee4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:31.355361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-378a846e-2f59-42f3-ab17-c0556e7ee4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:31.360684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:31.556343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:31.560058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-378a846e-2f59-42f3-ab17-c0556e7ee4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:31.568378 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:31.570056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:31.571929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-51667f15-3603-482a-8458-c9a60fb40178 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.232976 May 25 16:40:31.572493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 834/1688] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:31 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:31.572987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-378a846e-2f59-42f3-ab17-c0556e7ee4f8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.234754 May 25 16:40:31.573571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 855/1689] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:31 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:32.001164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-da562a9c-3f54-4c0b-b76d-6af1229cd775 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:32.001780 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-da562a9c-3f54-4c0b-b76d-6af1229cd775 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:32.008743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da562a9c-3f54-4c0b-b76d-6af1229cd775 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:32.009187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da562a9c-3f54-4c0b-b76d-6af1229cd775 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:32.009942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da562a9c-3f54-4c0b-b76d-6af1229cd775 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:32.010543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da562a9c-3f54-4c0b-b76d-6af1229cd775 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:32.073246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-da562a9c-3f54-4c0b-b76d-6af1229cd775 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:32.081735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-da562a9c-3f54-4c0b-b76d-6af1229cd775 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085017 May 25 16:40:32.082311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 835/1690] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:31 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:32.185223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-20856487-5244-4c84-bc13-241964e94c88 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:32.185849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-20856487-5244-4c84-bc13-241964e94c88 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:32.192813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20856487-5244-4c84-bc13-241964e94c88 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:32.193262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20856487-5244-4c84-bc13-241964e94c88 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:32.194019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20856487-5244-4c84-bc13-241964e94c88 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:32.194456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20856487-5244-4c84-bc13-241964e94c88 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:32.330991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-20856487-5244-4c84-bc13-241964e94c88 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:32.339264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-20856487-5244-4c84-bc13-241964e94c88 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.157967 May 25 16:40:32.339816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 856/1691] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:32 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:32.590544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:32.590850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-55d8e3f0-ab58-4928-a863-cbf13565cce8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:32.591122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:32.591524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-55d8e3f0-ab58-4928-a863-cbf13565cce8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:32.597920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:32.598125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-55d8e3f0-ab58-4928-a863-cbf13565cce8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:32.598367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:32.598479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-55d8e3f0-ab58-4928-a863-cbf13565cce8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:32.599139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-55d8e3f0-ab58-4928-a863-cbf13565cce8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:32.599346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:32.599612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:32.599725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-55d8e3f0-ab58-4928-a863-cbf13565cce8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:32.791545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-55d8e3f0-ab58-4928-a863-cbf13565cce8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:32.795498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:32.800054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-55d8e3f0-ab58-4928-a863-cbf13565cce8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.213128 May 25 16:40:32.800717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 857/1692] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:32 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:32.804281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:32.805780 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:32.806946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-51073adb-6843-4fac-8148-9057a0698031 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.220446 May 25 16:40:32.807508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 836/1693] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:32 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:33.098783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b8b9025e-0926-4b21-842e-5b4215132896 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:33.099499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b8b9025e-0926-4b21-842e-5b4215132896 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:33.105981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b8b9025e-0926-4b21-842e-5b4215132896 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:33.106429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b8b9025e-0926-4b21-842e-5b4215132896 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:33.107194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b8b9025e-0926-4b21-842e-5b4215132896 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:33.107656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b8b9025e-0926-4b21-842e-5b4215132896 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:33.175150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b8b9025e-0926-4b21-842e-5b4215132896 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:33.183790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b8b9025e-0926-4b21-842e-5b4215132896 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088867 May 25 16:40:33.184313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 858/1694] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:33 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:33.356043 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-361f429d-c45a-4018-b67f-f52aa24a1a8a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:33.356589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-361f429d-c45a-4018-b67f-f52aa24a1a8a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:33.364344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-361f429d-c45a-4018-b67f-f52aa24a1a8a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:33.365119 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-361f429d-c45a-4018-b67f-f52aa24a1a8a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:33.366361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-361f429d-c45a-4018-b67f-f52aa24a1a8a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:33.367017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-361f429d-c45a-4018-b67f-f52aa24a1a8a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:33.531292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-361f429d-c45a-4018-b67f-f52aa24a1a8a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:33.540022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-361f429d-c45a-4018-b67f-f52aa24a1a8a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.187715 May 25 16:40:33.540590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 837/1695] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:33 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:33.817512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a354d967-710c-4477-81f6-315a0c774d51 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:33.818107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a354d967-710c-4477-81f6-315a0c774d51 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:33.825133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:33.825330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a354d967-710c-4477-81f6-315a0c774d51 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:33.825737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:33.825904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a354d967-710c-4477-81f6-315a0c774d51 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:33.826549 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a354d967-710c-4477-81f6-315a0c774d51 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:33.826980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a354d967-710c-4477-81f6-315a0c774d51 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:33.832193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:33.832633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:33.833369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:33.833794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:34.026932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a354d967-710c-4477-81f6-315a0c774d51 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:34.035923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a354d967-710c-4477-81f6-315a0c774d51 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222262 May 25 16:40:34.036628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 859/1696] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:33 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:34.040582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:34.049460 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:34.050953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:34.052147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-749f244b-71bf-43bc-976d-d8148b5bc9c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.230976 May 25 16:40:34.052682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 838/1697] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:33 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:34.205601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-812670ce-3b05-4408-86c2-83d67ebe1e49 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:34.206613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-812670ce-3b05-4408-86c2-83d67ebe1e49 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:34.216780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-812670ce-3b05-4408-86c2-83d67ebe1e49 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:34.217521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-812670ce-3b05-4408-86c2-83d67ebe1e49 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:34.218780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-812670ce-3b05-4408-86c2-83d67ebe1e49 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:34.219551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-812670ce-3b05-4408-86c2-83d67ebe1e49 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:34.284326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-812670ce-3b05-4408-86c2-83d67ebe1e49 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:34.292910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-812670ce-3b05-4408-86c2-83d67ebe1e49 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093969 May 25 16:40:34.293481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 860/1698] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:34 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:34.561479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2a7862ca-94b0-4740-bb29-49d3b99176dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:34.562480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2a7862ca-94b0-4740-bb29-49d3b99176dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:34.573442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a7862ca-94b0-4740-bb29-49d3b99176dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:34.574195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a7862ca-94b0-4740-bb29-49d3b99176dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:34.575489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a7862ca-94b0-4740-bb29-49d3b99176dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:34.576232 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a7862ca-94b0-4740-bb29-49d3b99176dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:34.720671 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2a7862ca-94b0-4740-bb29-49d3b99176dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:34.733338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2a7862ca-94b0-4740-bb29-49d3b99176dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.178082 May 25 16:40:34.734213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 839/1699] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:34 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:35.057864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0f55ebb9-b1e1-4b98-8641-dcafc020e30e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:35.059322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0f55ebb9-b1e1-4b98-8641-dcafc020e30e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:35.070881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0f55ebb9-b1e1-4b98-8641-dcafc020e30e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:35.071705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0f55ebb9-b1e1-4b98-8641-dcafc020e30e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:35.073058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0f55ebb9-b1e1-4b98-8641-dcafc020e30e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:35.073493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0f55ebb9-b1e1-4b98-8641-dcafc020e30e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:35.075382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:35.075986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:35.083931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:35.084375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:35.085150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:35.085576 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:35.275196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0f55ebb9-b1e1-4b98-8641-dcafc020e30e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:35.278678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:35.283588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0f55ebb9-b1e1-4b98-8641-dcafc020e30e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.232144 May 25 16:40:35.284129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 861/1700] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:35 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:35.287418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:35.288943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:35.290410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8ed81e64-343f-4f4b-9c3c-1b7016d7ac88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.223108 May 25 16:40:35.290960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 840/1701] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:35 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:35.310322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9bd2313c-97b0-4b76-9b43-37c2c1c6af47 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:35.310927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9bd2313c-97b0-4b76-9b43-37c2c1c6af47 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:35.317331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9bd2313c-97b0-4b76-9b43-37c2c1c6af47 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:35.317774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9bd2313c-97b0-4b76-9b43-37c2c1c6af47 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:35.318519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9bd2313c-97b0-4b76-9b43-37c2c1c6af47 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:35.318953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9bd2313c-97b0-4b76-9b43-37c2c1c6af47 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:35.378958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9bd2313c-97b0-4b76-9b43-37c2c1c6af47 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:35.386870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9bd2313c-97b0-4b76-9b43-37c2c1c6af47 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.080349 May 25 16:40:35.387588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 862/1702] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:35 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:35.758143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-573afb4b-408d-4020-abf1-47af0be5c5e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:35.759227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-573afb4b-408d-4020-abf1-47af0be5c5e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:35.766731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-573afb4b-408d-4020-abf1-47af0be5c5e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:35.767180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-573afb4b-408d-4020-abf1-47af0be5c5e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:35.767978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-573afb4b-408d-4020-abf1-47af0be5c5e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:35.768408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-573afb4b-408d-4020-abf1-47af0be5c5e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:35.916570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-573afb4b-408d-4020-abf1-47af0be5c5e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:35.925139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-573afb4b-408d-4020-abf1-47af0be5c5e2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173405 May 25 16:40:35.925713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 841/1703] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:35 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:36.305504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6a13e43c-ea64-4dc9-a57a-e9d7f77f0eb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:36.306370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6a13e43c-ea64-4dc9-a57a-e9d7f77f0eb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:36.310166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:36.310771 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:36.313427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a13e43c-ea64-4dc9-a57a-e9d7f77f0eb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:36.313908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a13e43c-ea64-4dc9-a57a-e9d7f77f0eb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:36.314678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a13e43c-ea64-4dc9-a57a-e9d7f77f0eb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:36.315116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a13e43c-ea64-4dc9-a57a-e9d7f77f0eb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:36.316771 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:36.317220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:36.317970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:36.318396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:36.506127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6a13e43c-ea64-4dc9-a57a-e9d7f77f0eb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:36.510210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:36.514744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6a13e43c-ea64-4dc9-a57a-e9d7f77f0eb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.215458 May 25 16:40:36.515396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 863/1704] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:36 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:36.519478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:36.519903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-317a30e8-e41f-4524-aac4-3d5327310ce5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:36.520519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-317a30e8-e41f-4524-aac4-3d5327310ce5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:36.520973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:36.522121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-62859010-c2f7-4301-82d8-8f4118bd6208 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.216532 May 25 16:40:36.522830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 842/1705] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:36 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:36.526637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-317a30e8-e41f-4524-aac4-3d5327310ce5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:36.527036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-317a30e8-e41f-4524-aac4-3d5327310ce5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:36.527826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-317a30e8-e41f-4524-aac4-3d5327310ce5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:36.528265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-317a30e8-e41f-4524-aac4-3d5327310ce5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:36.597456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-317a30e8-e41f-4524-aac4-3d5327310ce5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:36.606064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-317a30e8-e41f-4524-aac4-3d5327310ce5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089738 May 25 16:40:36.606648 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 864/1706] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:36 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:36.946034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1468159e-5356-4e84-99cf-602d021c1c24 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:36.947020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1468159e-5356-4e84-99cf-602d021c1c24 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:36.958660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1468159e-5356-4e84-99cf-602d021c1c24 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:36.959597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1468159e-5356-4e84-99cf-602d021c1c24 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:36.960866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1468159e-5356-4e84-99cf-602d021c1c24 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:36.961664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1468159e-5356-4e84-99cf-602d021c1c24 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:37.095835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1468159e-5356-4e84-99cf-602d021c1c24 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:37.104711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1468159e-5356-4e84-99cf-602d021c1c24 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.165332 May 25 16:40:37.105281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 843/1707] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:36 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:37.535683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c75d5970-770d-4f46-9ba6-669de09f89a4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:37.536659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c75d5970-770d-4f46-9ba6-669de09f89a4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:37.544320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:37.545315 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:37.548101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c75d5970-770d-4f46-9ba6-669de09f89a4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:37.548832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c75d5970-770d-4f46-9ba6-669de09f89a4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:37.550555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c75d5970-770d-4f46-9ba6-669de09f89a4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:37.551350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c75d5970-770d-4f46-9ba6-669de09f89a4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:37.555326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:37.555924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:37.557231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:37.557694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:37.761531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c75d5970-770d-4f46-9ba6-669de09f89a4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:37.764302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:37.769743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c75d5970-770d-4f46-9ba6-669de09f89a4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.240538 May 25 16:40:37.770322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 865/1708] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:37 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:37.773798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:37.774677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4e89ba89-02d0-4d1b-ae33-ba9a8e57b81b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:37.775274 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4e89ba89-02d0-4d1b-ae33-ba9a8e57b81b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:37.775435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:37.776943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-45d144a4-4a8b-460c-a6ae-af5ee35bb9d1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.239296 May 25 16:40:37.777658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 844/1709] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:37 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:37.781346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4e89ba89-02d0-4d1b-ae33-ba9a8e57b81b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:37.781784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4e89ba89-02d0-4d1b-ae33-ba9a8e57b81b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:37.782534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4e89ba89-02d0-4d1b-ae33-ba9a8e57b81b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:37.782958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4e89ba89-02d0-4d1b-ae33-ba9a8e57b81b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:37.845568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4e89ba89-02d0-4d1b-ae33-ba9a8e57b81b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:37.857934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4e89ba89-02d0-4d1b-ae33-ba9a8e57b81b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.086275 May 25 16:40:37.858721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 866/1710] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:37 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:38.125222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-120166da-57c0-4818-9010-08457673b613 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:38.125740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-120166da-57c0-4818-9010-08457673b613 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:38.135586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120166da-57c0-4818-9010-08457673b613 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:38.136031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120166da-57c0-4818-9010-08457673b613 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:38.136782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120166da-57c0-4818-9010-08457673b613 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:38.137272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-120166da-57c0-4818-9010-08457673b613 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:38.139886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:38.282623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-120166da-57c0-4818-9010-08457673b613 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:38.291672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-120166da-57c0-4818-9010-08457673b613 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172521 May 25 16:40:38.292242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 845/1711] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:38 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:38.791609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-46ed6ab0-315a-41ee-997d-4f0f2fb60631 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:38.792399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-46ed6ab0-315a-41ee-997d-4f0f2fb60631 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:38.796334 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:38.796944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:38.800192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-46ed6ab0-315a-41ee-997d-4f0f2fb60631 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:38.800631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-46ed6ab0-315a-41ee-997d-4f0f2fb60631 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:38.801406 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-46ed6ab0-315a-41ee-997d-4f0f2fb60631 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:38.801847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-46ed6ab0-315a-41ee-997d-4f0f2fb60631 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:38.803252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:38.803665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:38.804454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:38.804886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:39.005081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-46ed6ab0-315a-41ee-997d-4f0f2fb60631 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:39.009140 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:39.013656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-46ed6ab0-315a-41ee-997d-4f0f2fb60631 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.228925 May 25 16:40:39.014213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 867/1712] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:38 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:39.018029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:39.018655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8b25adbd-1b30-4812-8182-a8ec5e464f2d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:39.019295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8b25adbd-1b30-4812-8182-a8ec5e464f2d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:39.019526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:39.020776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4ef22b32-c7c0-426c-a062-b542c1ea9fb0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.228408 May 25 16:40:39.021402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 846/1713] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:38 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:39.025254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b25adbd-1b30-4812-8182-a8ec5e464f2d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:39.025766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b25adbd-1b30-4812-8182-a8ec5e464f2d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:39.026523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b25adbd-1b30-4812-8182-a8ec5e464f2d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:39.026960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b25adbd-1b30-4812-8182-a8ec5e464f2d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:39.087946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8b25adbd-1b30-4812-8182-a8ec5e464f2d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:39.096390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8b25adbd-1b30-4812-8182-a8ec5e464f2d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.081192 May 25 16:40:39.096966 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 868/1714] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:39 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:39.312907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f0377edf-f140-445f-ab55-0a090ba4eabd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:39.313935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f0377edf-f140-445f-ab55-0a090ba4eabd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:39.325130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f0377edf-f140-445f-ab55-0a090ba4eabd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:39.325670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f0377edf-f140-445f-ab55-0a090ba4eabd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:39.326919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f0377edf-f140-445f-ab55-0a090ba4eabd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:39.327691 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f0377edf-f140-445f-ab55-0a090ba4eabd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:39.464616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f0377edf-f140-445f-ab55-0a090ba4eabd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:39.472966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f0377edf-f140-445f-ab55-0a090ba4eabd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.166572 May 25 16:40:39.473536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 847/1715] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:39 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:40.035728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6a2e89c3-8be4-4b6c-a1e3-38ebe67a9d7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:40.036942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6a2e89c3-8be4-4b6c-a1e3-38ebe67a9d7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:40.042873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:40.043441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:40.046429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=98 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:40.048528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a2e89c3-8be4-4b6c-a1e3-38ebe67a9d7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:40.049317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a2e89c3-8be4-4b6c-a1e3-38ebe67a9d7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:40.050723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a2e89c3-8be4-4b6c-a1e3-38ebe67a9d7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:40.051496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6a2e89c3-8be4-4b6c-a1e3-38ebe67a9d7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:40.055440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:40.056217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:40.057379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:40.057800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:40.245095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6a2e89c3-8be4-4b6c-a1e3-38ebe67a9d7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:40.248121 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:40.253502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6a2e89c3-8be4-4b6c-a1e3-38ebe67a9d7a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.224440 May 25 16:40:40.254169 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 869/1716] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:40 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:40.256704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:40.258196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:40.258653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0036a20a-f492-4cc4-8142-f5ff0cf8b9ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:40.259280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0036a20a-f492-4cc4-8142-f5ff0cf8b9ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:40.259442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e0d86819-8d81-4c55-b4bd-359738bfc260 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.223972 May 25 16:40:40.260095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 848/1717] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:40 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:40.266297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0036a20a-f492-4cc4-8142-f5ff0cf8b9ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:40.266739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0036a20a-f492-4cc4-8142-f5ff0cf8b9ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:40.267616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0036a20a-f492-4cc4-8142-f5ff0cf8b9ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:40.268073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0036a20a-f492-4cc4-8142-f5ff0cf8b9ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:40.330712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0036a20a-f492-4cc4-8142-f5ff0cf8b9ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:40.342222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0036a20a-f492-4cc4-8142-f5ff0cf8b9ea tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.086870 May 25 16:40:40.342956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 870/1718] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:40 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:40.493822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-635e57ed-b545-478f-a07d-e55e36a5c821 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:40.494822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-635e57ed-b545-478f-a07d-e55e36a5c821 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:40.505858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-635e57ed-b545-478f-a07d-e55e36a5c821 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:40.506608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-635e57ed-b545-478f-a07d-e55e36a5c821 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:40.507918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-635e57ed-b545-478f-a07d-e55e36a5c821 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:40.508633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-635e57ed-b545-478f-a07d-e55e36a5c821 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:40.646314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-635e57ed-b545-478f-a07d-e55e36a5c821 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:40.654520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-635e57ed-b545-478f-a07d-e55e36a5c821 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.167110 May 25 16:40:40.655087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 849/1719] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:40 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:41.275223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-042c7f3c-29da-4f3f-9206-7cd725d3a369 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:41.276087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-042c7f3c-29da-4f3f-9206-7cd725d3a369 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:41.280326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:41.280926 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:41.282935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-042c7f3c-29da-4f3f-9206-7cd725d3a369 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:41.283417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-042c7f3c-29da-4f3f-9206-7cd725d3a369 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:41.284183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-042c7f3c-29da-4f3f-9206-7cd725d3a369 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:41.284632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-042c7f3c-29da-4f3f-9206-7cd725d3a369 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:41.287131 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:41.287607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:41.288343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:41.288787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:41.476912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-042c7f3c-29da-4f3f-9206-7cd725d3a369 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:41.479175 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:41.480092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:41.486426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-042c7f3c-29da-4f3f-9206-7cd725d3a369 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217843 May 25 16:40:41.486996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 871/1720] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:41 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:41.489151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:41.490658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:41.491539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c349662e-08cf-4d93-8c4e-130e7836644f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:41.491967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6d3c3c50-f121-402a-b639-3138a2ed7bb6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.217003 May 25 16:40:41.492143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c349662e-08cf-4d93-8c4e-130e7836644f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:41.492674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 850/1721] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:41.498319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c349662e-08cf-4d93-8c4e-130e7836644f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:41.498771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c349662e-08cf-4d93-8c4e-130e7836644f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:41.499561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c349662e-08cf-4d93-8c4e-130e7836644f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:41.499992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c349662e-08cf-4d93-8c4e-130e7836644f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:41.560640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c349662e-08cf-4d93-8c4e-130e7836644f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:41.568977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c349662e-08cf-4d93-8c4e-130e7836644f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.080992 May 25 16:40:41.569559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 872/1722] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:41 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:41.675867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-59bbfe43-ca8d-4a7f-bae5-a548696c52f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:41.676721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-59bbfe43-ca8d-4a7f-bae5-a548696c52f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:41.687680 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-59bbfe43-ca8d-4a7f-bae5-a548696c52f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:41.688432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-59bbfe43-ca8d-4a7f-bae5-a548696c52f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:41.689702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-59bbfe43-ca8d-4a7f-bae5-a548696c52f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:41.690406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-59bbfe43-ca8d-4a7f-bae5-a548696c52f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:41.835600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-59bbfe43-ca8d-4a7f-bae5-a548696c52f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:41.845429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-59bbfe43-ca8d-4a7f-bae5-a548696c52f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.176150 May 25 16:40:41.846005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 851/1723] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:41 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:42.507273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4d94ac95-ed9f-44f4-90ca-a63e44a0f231 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:42.508117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4d94ac95-ed9f-44f4-90ca-a63e44a0f231 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:42.512076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:42.512708 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:42.515684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d94ac95-ed9f-44f4-90ca-a63e44a0f231 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:42.516124 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d94ac95-ed9f-44f4-90ca-a63e44a0f231 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:42.516883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d94ac95-ed9f-44f4-90ca-a63e44a0f231 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:42.517308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4d94ac95-ed9f-44f4-90ca-a63e44a0f231 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:42.519468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:42.519922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:42.520674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:42.521109 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:42.708427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4d94ac95-ed9f-44f4-90ca-a63e44a0f231 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:42.711279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:42.716754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4d94ac95-ed9f-44f4-90ca-a63e44a0f231 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.215575 May 25 16:40:42.717385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 873/1724] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:42 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:42.720723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:42.721935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-370b0290-de59-40d7-9391-29dbfab63f81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:42.722227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:42.722681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-370b0290-de59-40d7-9391-29dbfab63f81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:42.723448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2b9c07a6-93c6-4879-8544-a04a09201689 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.216571 May 25 16:40:42.724335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 852/1725] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:42.728851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-370b0290-de59-40d7-9391-29dbfab63f81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:42.729295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-370b0290-de59-40d7-9391-29dbfab63f81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:42.730084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-370b0290-de59-40d7-9391-29dbfab63f81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:42.730514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-370b0290-de59-40d7-9391-29dbfab63f81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:42.794114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-370b0290-de59-40d7-9391-29dbfab63f81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:42.802249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-370b0290-de59-40d7-9391-29dbfab63f81 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083830 May 25 16:40:42.802822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 874/1726] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:42 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:42.866878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3e3aa20b-6797-4753-a5c3-7985d4aedd27 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:42.867676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3e3aa20b-6797-4753-a5c3-7985d4aedd27 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:42.874295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e3aa20b-6797-4753-a5c3-7985d4aedd27 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:42.874730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e3aa20b-6797-4753-a5c3-7985d4aedd27 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:42.875534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e3aa20b-6797-4753-a5c3-7985d4aedd27 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:42.875973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e3aa20b-6797-4753-a5c3-7985d4aedd27 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:43.007685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3e3aa20b-6797-4753-a5c3-7985d4aedd27 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:43.020073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3e3aa20b-6797-4753-a5c3-7985d4aedd27 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.157719 May 25 16:40:43.020817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 853/1727] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:42 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:43.738581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d863da4e-8a5d-47d7-af21-86f2ef42e06b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:43.739569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d863da4e-8a5d-47d7-af21-86f2ef42e06b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:43.745617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:43.746258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:43.748251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d863da4e-8a5d-47d7-af21-86f2ef42e06b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:43.748703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d863da4e-8a5d-47d7-af21-86f2ef42e06b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:43.749467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d863da4e-8a5d-47d7-af21-86f2ef42e06b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:43.749897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d863da4e-8a5d-47d7-af21-86f2ef42e06b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:43.753595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:43.754047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:43.754943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:43.755459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:43.943965 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d863da4e-8a5d-47d7-af21-86f2ef42e06b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:43.947965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:43.953150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d863da4e-8a5d-47d7-af21-86f2ef42e06b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221347 May 25 16:40:43.953770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 875/1728] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:43 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:43.957418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:43.958448 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2ddb13ff-f362-46fb-b5cd-7b665ce16c2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:43.959045 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2ddb13ff-f362-46fb-b5cd-7b665ce16c2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:43.959208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:43.960355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6bb149bc-6edd-4b33-85d0-44424093365c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.221393 May 25 16:40:43.960973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 854/1729] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:43.970062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2ddb13ff-f362-46fb-b5cd-7b665ce16c2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:43.970522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2ddb13ff-f362-46fb-b5cd-7b665ce16c2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:43.971395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2ddb13ff-f362-46fb-b5cd-7b665ce16c2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:43.971895 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2ddb13ff-f362-46fb-b5cd-7b665ce16c2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:44.034240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2ddb13ff-f362-46fb-b5cd-7b665ce16c2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:44.039755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-811c63c8-be83-426e-aebc-acb60acad515 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:44.040365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-811c63c8-be83-426e-aebc-acb60acad515 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:44.043927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2ddb13ff-f362-46fb-b5cd-7b665ce16c2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089144 May 25 16:40:44.044496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 876/1730] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:43 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:44.046668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-811c63c8-be83-426e-aebc-acb60acad515 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:44.047120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-811c63c8-be83-426e-aebc-acb60acad515 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:44.047925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-811c63c8-be83-426e-aebc-acb60acad515 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:44.048364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-811c63c8-be83-426e-aebc-acb60acad515 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:44.064743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:40:44.064923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:40:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b (ip 10.222.0.45) !!! May 25 16:40:44.191317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-811c63c8-be83-426e-aebc-acb60acad515 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:44.200036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-811c63c8-be83-426e-aebc-acb60acad515 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.164794 May 25 16:40:44.200599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 855/1731] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:44 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:44.973833 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0e3c52cd-fef5-4407-aa0e-464448e1f35f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:44.975005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0e3c52cd-fef5-4407-aa0e-464448e1f35f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:44.981875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:44.982586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:44.983348 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e3c52cd-fef5-4407-aa0e-464448e1f35f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:44.983796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e3c52cd-fef5-4407-aa0e-464448e1f35f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:44.984555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e3c52cd-fef5-4407-aa0e-464448e1f35f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:44.984983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e3c52cd-fef5-4407-aa0e-464448e1f35f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:44.989459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:44.989937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:44.990775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:44.991271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:45.183125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:45.186202 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0e3c52cd-fef5-4407-aa0e-464448e1f35f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:45.192052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:45.193553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:45.194722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c78e0433-7012-474d-b584-234c28310bc8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.218556 May 25 16:40:45.195301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 856/1732] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:44 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:45.195429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0e3c52cd-fef5-4407-aa0e-464448e1f35f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.227973 May 25 16:40:45.196185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 877/1733] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:44 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:45.200270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ac1ad77d-b021-4b7e-a3c8-907d2f2352bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:45.200860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ac1ad77d-b021-4b7e-a3c8-907d2f2352bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:45.207193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac1ad77d-b021-4b7e-a3c8-907d2f2352bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:45.207639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac1ad77d-b021-4b7e-a3c8-907d2f2352bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:45.208405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac1ad77d-b021-4b7e-a3c8-907d2f2352bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:45.208834 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac1ad77d-b021-4b7e-a3c8-907d2f2352bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:45.217445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3daeff76-4a3b-4233-bb25-7e3f18ea50a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:45.218054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3daeff76-4a3b-4233-bb25-7e3f18ea50a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:45.224125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3daeff76-4a3b-4233-bb25-7e3f18ea50a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:45.224508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3daeff76-4a3b-4233-bb25-7e3f18ea50a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:45.225268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3daeff76-4a3b-4233-bb25-7e3f18ea50a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:45.225707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3daeff76-4a3b-4233-bb25-7e3f18ea50a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:45.277236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ac1ad77d-b021-4b7e-a3c8-907d2f2352bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:45.285985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ac1ad77d-b021-4b7e-a3c8-907d2f2352bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089299 May 25 16:40:45.286564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 857/1734] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:45 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:45.371125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3daeff76-4a3b-4233-bb25-7e3f18ea50a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:45.379492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3daeff76-4a3b-4233-bb25-7e3f18ea50a5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.166009 May 25 16:40:45.379998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 878/1735] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:45 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:40:46.216786 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fb64379b-f0b6-46a5-86b9-3638ce613062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:46.217682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:46.217682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:46.218463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fb64379b-f0b6-46a5-86b9-3638ce613062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:46.228419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fb64379b-f0b6-46a5-86b9-3638ce613062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:46.228817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:46.229077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fb64379b-f0b6-46a5-86b9-3638ce613062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:46.229236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:46.229831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fb64379b-f0b6-46a5-86b9-3638ce613062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:46.229980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:46.230258 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:46.230434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fb64379b-f0b6-46a5-86b9-3638ce613062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:46.424034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:46.426955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fb64379b-f0b6-46a5-86b9-3638ce613062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:46.432792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:46.434286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:46.435480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-60a72d78-b737-40eb-b6ce-ef9ae64f0e08 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.225278 May 25 16:40:46.436038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 879/1736] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:46 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:46.437380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fb64379b-f0b6-46a5-86b9-3638ce613062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.227160 May 25 16:40:46.447094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 858/1737] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:46 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:46.447375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e1d2df7e-5c9a-4c9d-b774-5a5bc9617326 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:46.447375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e1d2df7e-5c9a-4c9d-b774-5a5bc9617326 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:46.447568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-000dd76c-72e3-4ad9-9abb-c90d8913e33c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:46.447568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-000dd76c-72e3-4ad9-9abb-c90d8913e33c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:46.447772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1d2df7e-5c9a-4c9d-b774-5a5bc9617326 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:46.447772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1d2df7e-5c9a-4c9d-b774-5a5bc9617326 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:46.448416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1d2df7e-5c9a-4c9d-b774-5a5bc9617326 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:46.448857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1d2df7e-5c9a-4c9d-b774-5a5bc9617326 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:46.450479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-000dd76c-72e3-4ad9-9abb-c90d8913e33c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:46.450929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-000dd76c-72e3-4ad9-9abb-c90d8913e33c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:46.451726 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-000dd76c-72e3-4ad9-9abb-c90d8913e33c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:46.452162 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-000dd76c-72e3-4ad9-9abb-c90d8913e33c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:46.519774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e1d2df7e-5c9a-4c9d-b774-5a5bc9617326 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:46.528156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e1d2df7e-5c9a-4c9d-b774-5a5bc9617326 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091192 May 25 16:40:46.528763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 880/1738] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:46 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:46.603508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-000dd76c-72e3-4ad9-9abb-c90d8913e33c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:46.612138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-000dd76c-72e3-4ad9-9abb-c90d8913e33c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173203 May 25 16:40:46.612741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 859/1739] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:46 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:47.458383 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-eb40d0db-454a-4a9e-bab2-430e999efccd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:47.459273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-eb40d0db-454a-4a9e-bab2-430e999efccd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:47.462613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:47.463356 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=96 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:47.466974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:47.468352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eb40d0db-454a-4a9e-bab2-430e999efccd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:47.468807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eb40d0db-454a-4a9e-bab2-430e999efccd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:47.469645 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eb40d0db-454a-4a9e-bab2-430e999efccd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:47.470081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eb40d0db-454a-4a9e-bab2-430e999efccd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:47.476567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:47.477012 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:47.477777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:47.478208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:47.673176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-eb40d0db-454a-4a9e-bab2-430e999efccd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:47.674816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:47.682591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-eb40d0db-454a-4a9e-bab2-430e999efccd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.231008 May 25 16:40:47.683194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 881/1740] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:47 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:47.683822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:47.685334 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:47.686672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-578a330e-1738-4b5a-b580-c1ecbb817bc0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.230679 May 25 16:40:47.693106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 860/1741] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:47.693287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a4af4e37-7fff-424b-add0-f85496462333 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:47.693287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a4af4e37-7fff-424b-add0-f85496462333 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:47.693520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-218d9cb3-97fd-4b90-bd5f-1033b06cb8b5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:47.693520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-218d9cb3-97fd-4b90-bd5f-1033b06cb8b5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:47.694863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4af4e37-7fff-424b-add0-f85496462333 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:47.695443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4af4e37-7fff-424b-add0-f85496462333 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:47.696236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4af4e37-7fff-424b-add0-f85496462333 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:47.696680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4af4e37-7fff-424b-add0-f85496462333 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:47.699418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-218d9cb3-97fd-4b90-bd5f-1033b06cb8b5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:47.699876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-218d9cb3-97fd-4b90-bd5f-1033b06cb8b5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:47.700627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-218d9cb3-97fd-4b90-bd5f-1033b06cb8b5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:47.701062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-218d9cb3-97fd-4b90-bd5f-1033b06cb8b5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:47.764469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a4af4e37-7fff-424b-add0-f85496462333 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:47.772459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a4af4e37-7fff-424b-add0-f85496462333 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088356 May 25 16:40:47.773035 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 882/1742] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:47 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:47.854018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-218d9cb3-97fd-4b90-bd5f-1033b06cb8b5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:47.856335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:47.863495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-218d9cb3-97fd-4b90-bd5f-1033b06cb8b5 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.175251 May 25 16:40:47.864052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 861/1743] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:47 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:48.704252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bd4714c6-a8ab-4d60-825e-f2fd99b1be74 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:48.705097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bd4714c6-a8ab-4d60-825e-f2fd99b1be74 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:48.708934 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:48.710220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:48.716372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd4714c6-a8ab-4d60-825e-f2fd99b1be74 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:48.717110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd4714c6-a8ab-4d60-825e-f2fd99b1be74 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:48.718363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd4714c6-a8ab-4d60-825e-f2fd99b1be74 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:48.719085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bd4714c6-a8ab-4d60-825e-f2fd99b1be74 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:48.721276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:48.722030 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:48.723425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:48.724156 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:48.918336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:48.922958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bd4714c6-a8ab-4d60-825e-f2fd99b1be74 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:48.931801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:48.933425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:48.933993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bd4714c6-a8ab-4d60-825e-f2fd99b1be74 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.236922 May 25 16:40:48.934569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 883/1744] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:48 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:48.934917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c28d7328-99b4-4eb4-b271-a49fc651099a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.232224 May 25 16:40:48.935756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 862/1745] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:48.939973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bae617eb-e69c-499f-8e43-c395a65b78d3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:48.940650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bae617eb-e69c-499f-8e43-c395a65b78d3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:48.946601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bae617eb-e69c-499f-8e43-c395a65b78d3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:48.947017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bae617eb-e69c-499f-8e43-c395a65b78d3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:48.947205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-85573d95-347a-4d0d-9e91-d8b5cc5e7a3e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:48.947788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bae617eb-e69c-499f-8e43-c395a65b78d3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:48.947916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-85573d95-347a-4d0d-9e91-d8b5cc5e7a3e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:48.948213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bae617eb-e69c-499f-8e43-c395a65b78d3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:48.954399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85573d95-347a-4d0d-9e91-d8b5cc5e7a3e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:48.954827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85573d95-347a-4d0d-9e91-d8b5cc5e7a3e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:48.955710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85573d95-347a-4d0d-9e91-d8b5cc5e7a3e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:48.956181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85573d95-347a-4d0d-9e91-d8b5cc5e7a3e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:49.274010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-85573d95-347a-4d0d-9e91-d8b5cc5e7a3e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:49.282744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-85573d95-347a-4d0d-9e91-d8b5cc5e7a3e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.347152 May 25 16:40:49.283369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 884/1746] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:48 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:49.336693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bae617eb-e69c-499f-8e43-c395a65b78d3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:49.345099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bae617eb-e69c-499f-8e43-c395a65b78d3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.408531 May 25 16:40:49.345678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 863/1747] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:48 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:49.955729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:49.956539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:49.959040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-02222ff0-e196-4831-95cc-d9713416e197 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:49.959820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-02222ff0-e196-4831-95cc-d9713416e197 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:49.964648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:49.965183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:49.966059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:49.966592 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:49.967786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02222ff0-e196-4831-95cc-d9713416e197 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:49.968271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02222ff0-e196-4831-95cc-d9713416e197 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:49.969192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02222ff0-e196-4831-95cc-d9713416e197 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:49.969706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-02222ff0-e196-4831-95cc-d9713416e197 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:50.159489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:50.161999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-02222ff0-e196-4831-95cc-d9713416e197 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:50.169105 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:50.170453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-02222ff0-e196-4831-95cc-d9713416e197 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221319 May 25 16:40:50.170799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:50.173913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 885/1748] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:49 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:50.174100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3d66a7d1-9a44-477f-a169-be51c64f2bc9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.220995 May 25 16:40:50.174100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 864/1749] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:50.305748 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9e69e2ae-2e0c-4700-a9b3-69d08e69fc2e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:50.306858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9e69e2ae-2e0c-4700-a9b3-69d08e69fc2e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:50.313054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e69e2ae-2e0c-4700-a9b3-69d08e69fc2e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:50.313505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e69e2ae-2e0c-4700-a9b3-69d08e69fc2e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:50.314259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e69e2ae-2e0c-4700-a9b3-69d08e69fc2e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:50.314693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9e69e2ae-2e0c-4700-a9b3-69d08e69fc2e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:50.362356 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4c0cdb06-1977-4dfc-940f-380f19901230 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:50.362979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4c0cdb06-1977-4dfc-940f-380f19901230 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:50.369709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c0cdb06-1977-4dfc-940f-380f19901230 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:50.370153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c0cdb06-1977-4dfc-940f-380f19901230 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:50.370921 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c0cdb06-1977-4dfc-940f-380f19901230 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:50.371646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c0cdb06-1977-4dfc-940f-380f19901230 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:50.545670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4c0cdb06-1977-4dfc-940f-380f19901230 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:50.557983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4c0cdb06-1977-4dfc-940f-380f19901230 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.199544 May 25 16:40:50.558572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 865/1750] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:50 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:50.565928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9e69e2ae-2e0c-4700-a9b3-69d08e69fc2e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:50.574350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9e69e2ae-2e0c-4700-a9b3-69d08e69fc2e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.275009 May 25 16:40:50.574908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 886/1751] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:50 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:51.192623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4d877d20-1b13-4938-a165-a7d3371e662f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:51.193192 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4d877d20-1b13-4938-a165-a7d3371e662f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:51.194396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:51.195372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:51.202680 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d877d20-1b13-4938-a165-a7d3371e662f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:51.203610 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d877d20-1b13-4938-a165-a7d3371e662f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:51.204509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d877d20-1b13-4938-a165-a7d3371e662f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:51.205208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d877d20-1b13-4938-a165-a7d3371e662f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:51.206076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:51.207556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:51.208756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:51.209405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:51.417210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4d877d20-1b13-4938-a165-a7d3371e662f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:51.420280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:51.425878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4d877d20-1b13-4938-a165-a7d3371e662f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.240390 May 25 16:40:51.426436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 866/1752] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:51 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:51.429217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:51.430721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:51.431911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-acf963e9-4beb-48cd-a22d-ac87bbe7cbb2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.244357 May 25 16:40:51.432420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 887/1753] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:51.580515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-464b76a3-f3f4-4f62-aae9-aca87beddf4c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:51.581448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-464b76a3-f3f4-4f62-aae9-aca87beddf4c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:51.593397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-464b76a3-f3f4-4f62-aae9-aca87beddf4c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:51.593397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-464b76a3-f3f4-4f62-aae9-aca87beddf4c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:51.593820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-464b76a3-f3f4-4f62-aae9-aca87beddf4c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:51.594373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-464b76a3-f3f4-4f62-aae9-aca87beddf4c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:51.600795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c9257ef5-fe81-48bd-9fcf-d7f34e1b9a84 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:51.601797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c9257ef5-fe81-48bd-9fcf-d7f34e1b9a84 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:51.614038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c9257ef5-fe81-48bd-9fcf-d7f34e1b9a84 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:51.614733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c9257ef5-fe81-48bd-9fcf-d7f34e1b9a84 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:51.616036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c9257ef5-fe81-48bd-9fcf-d7f34e1b9a84 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:51.616718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c9257ef5-fe81-48bd-9fcf-d7f34e1b9a84 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:51.642177 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:51.710980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c9257ef5-fe81-48bd-9fcf-d7f34e1b9a84 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:51.719400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c9257ef5-fe81-48bd-9fcf-d7f34e1b9a84 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.127882 May 25 16:40:51.719966 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 888/1754] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:51 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:51.773954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-464b76a3-f3f4-4f62-aae9-aca87beddf4c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:51.782563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-464b76a3-f3f4-4f62-aae9-aca87beddf4c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.208521 May 25 16:40:51.783137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 867/1755] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:51 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:52.447602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-31b55584-a1b8-4b87-a62a-caef16b74507 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:52.448122 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-31b55584-a1b8-4b87-a62a-caef16b74507 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:52.453885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:52.454488 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:52.455068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31b55584-a1b8-4b87-a62a-caef16b74507 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:52.455549 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31b55584-a1b8-4b87-a62a-caef16b74507 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:52.456309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31b55584-a1b8-4b87-a62a-caef16b74507 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:52.456738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-31b55584-a1b8-4b87-a62a-caef16b74507 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:52.461007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:52.461456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:52.462211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:52.462643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:52.647184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-31b55584-a1b8-4b87-a62a-caef16b74507 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:52.650001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:52.655623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-31b55584-a1b8-4b87-a62a-caef16b74507 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.214575 May 25 16:40:52.656211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 889/1756] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:52 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:52.658807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:52.660330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:52.661480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ec455f02-e3af-4602-a6ed-9b542c7ad8ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.211544 May 25 16:40:52.662019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 868/1757] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:52.736673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fe68611c-2f33-4dfe-bf8d-eeffc799e126 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:52.737201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fe68611c-2f33-4dfe-bf8d-eeffc799e126 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:52.743598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe68611c-2f33-4dfe-bf8d-eeffc799e126 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:52.744135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe68611c-2f33-4dfe-bf8d-eeffc799e126 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:52.744904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe68611c-2f33-4dfe-bf8d-eeffc799e126 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:52.745342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fe68611c-2f33-4dfe-bf8d-eeffc799e126 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:52.800591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-efdc830d-9651-4364-8283-7ec2427ad5cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:52.801224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-efdc830d-9651-4364-8283-7ec2427ad5cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:52.807468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-efdc830d-9651-4364-8283-7ec2427ad5cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:52.807920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-efdc830d-9651-4364-8283-7ec2427ad5cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:52.808675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-efdc830d-9651-4364-8283-7ec2427ad5cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:52.809128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-efdc830d-9651-4364-8283-7ec2427ad5cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:52.813083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fe68611c-2f33-4dfe-bf8d-eeffc799e126 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:52.821857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fe68611c-2f33-4dfe-bf8d-eeffc799e126 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089174 May 25 16:40:52.822424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 890/1758] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:52 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:52.938959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-efdc830d-9651-4364-8283-7ec2427ad5cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:52.947642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-efdc830d-9651-4364-8283-7ec2427ad5cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.150709 May 25 16:40:52.948202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 869/1759] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:52 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:53.676686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-db318c81-17ef-44f2-a687-e26a67f98660 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:53.677774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-db318c81-17ef-44f2-a687-e26a67f98660 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:53.689236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db318c81-17ef-44f2-a687-e26a67f98660 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:53.690071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db318c81-17ef-44f2-a687-e26a67f98660 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:53.690415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:53.691396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db318c81-17ef-44f2-a687-e26a67f98660 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:53.691589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:53.692125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db318c81-17ef-44f2-a687-e26a67f98660 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:53.701003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:53.701456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:53.702218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:53.702652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:53.880448 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-db318c81-17ef-44f2-a687-e26a67f98660 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:53.889505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-db318c81-17ef-44f2-a687-e26a67f98660 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.219447 May 25 16:40:53.891439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 891/1760] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:53 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:53.895262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-08f8f51e-0ba5-4f6f-bb1b-b1a2b4955516 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:53.895874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-08f8f51e-0ba5-4f6f-bb1b-b1a2b4955516 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:53.901399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:53.903754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-08f8f51e-0ba5-4f6f-bb1b-b1a2b4955516 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:53.904204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-08f8f51e-0ba5-4f6f-bb1b-b1a2b4955516 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:53.904956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-08f8f51e-0ba5-4f6f-bb1b-b1a2b4955516 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:53.905419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-08f8f51e-0ba5-4f6f-bb1b-b1a2b4955516 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:53.910007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:53.911591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:53.912794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5a12eeb5-a701-4f57-9b14-f19d7e66cc43 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.235360 May 25 16:40:53.913389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 870/1761] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:53 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:53.965134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0e25920a-bd50-421e-89b7-49fdfb54f719 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:53.966880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0e25920a-bd50-421e-89b7-49fdfb54f719 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:53.970137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-08f8f51e-0ba5-4f6f-bb1b-b1a2b4955516 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:53.973017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e25920a-bd50-421e-89b7-49fdfb54f719 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:53.973473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e25920a-bd50-421e-89b7-49fdfb54f719 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:53.974256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e25920a-bd50-421e-89b7-49fdfb54f719 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:53.974774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e25920a-bd50-421e-89b7-49fdfb54f719 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:53.978631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-08f8f51e-0ba5-4f6f-bb1b-b1a2b4955516 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.087377 May 25 16:40:53.979237 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 892/1762] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:53 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:54.115648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0e25920a-bd50-421e-89b7-49fdfb54f719 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:54.125291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0e25920a-bd50-421e-89b7-49fdfb54f719 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.164156 May 25 16:40:54.125876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 871/1763] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:53 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:40:54.910275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-12f585bf-063d-477a-bfc9-011e24d96656 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:54.911312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-12f585bf-063d-477a-bfc9-011e24d96656 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:54.922867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12f585bf-063d-477a-bfc9-011e24d96656 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:54.923680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12f585bf-063d-477a-bfc9-011e24d96656 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:54.924948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12f585bf-063d-477a-bfc9-011e24d96656 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:54.925692 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-12f585bf-063d-477a-bfc9-011e24d96656 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:54.934009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:54.934621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:54.941430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:54.941875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:54.942634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:54.943066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:55.128405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-12f585bf-063d-477a-bfc9-011e24d96656 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:55.132099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:55.137062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-12f585bf-063d-477a-bfc9-011e24d96656 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.233189 May 25 16:40:55.137639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 893/1764] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:54 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:55.141829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:55.142209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e9a148f6-0e29-468a-b807-b1dd533d2dc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:55.142849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e9a148f6-0e29-468a-b807-b1dd533d2dc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:55.143352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:55.144792 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e6e38ac1-0d1f-4533-865c-eb82921815ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.215910 May 25 16:40:55.145516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 872/1765] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:55.149146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e9a148f6-0e29-468a-b807-b1dd533d2dc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:55.149613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e9a148f6-0e29-468a-b807-b1dd533d2dc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:55.149755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5b25bd72-21ea-433b-9bec-3a5cb5b151fd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:55.150309 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5b25bd72-21ea-433b-9bec-3a5cb5b151fd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:55.150439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e9a148f6-0e29-468a-b807-b1dd533d2dc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:55.150802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e9a148f6-0e29-468a-b807-b1dd533d2dc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:55.156865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5b25bd72-21ea-433b-9bec-3a5cb5b151fd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:55.157379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5b25bd72-21ea-433b-9bec-3a5cb5b151fd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:55.158233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5b25bd72-21ea-433b-9bec-3a5cb5b151fd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:55.158701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5b25bd72-21ea-433b-9bec-3a5cb5b151fd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:55.239359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e9a148f6-0e29-468a-b807-b1dd533d2dc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:55.247564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e9a148f6-0e29-468a-b807-b1dd533d2dc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.108931 May 25 16:40:55.248177 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 894/1766] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:55 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:55.311350 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5b25bd72-21ea-433b-9bec-3a5cb5b151fd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:55.320977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5b25bd72-21ea-433b-9bec-3a5cb5b151fd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.174659 May 25 16:40:55.321547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 873/1767] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:55 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:56.167352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:56.167893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0eae4c1f-dc98-469e-856e-b287842fee5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:56.168252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:56.168373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0eae4c1f-dc98-469e-856e-b287842fee5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:56.175285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eae4c1f-dc98-469e-856e-b287842fee5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:56.175484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:56.175727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eae4c1f-dc98-469e-856e-b287842fee5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:56.175907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:56.176491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eae4c1f-dc98-469e-856e-b287842fee5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:56.176656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:56.176903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0eae4c1f-dc98-469e-856e-b287842fee5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:56.177037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:56.369041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:56.373545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0eae4c1f-dc98-469e-856e-b287842fee5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:56.379664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:56.381400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:56.382189 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0eae4c1f-dc98-469e-856e-b287842fee5b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220545 May 25 16:40:56.382764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-86d3fa8b-e864-4167-8a2e-ded5a4cbbd87 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.222226 May 25 16:40:56.382876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 874/1768] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:56 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:56.383396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 895/1769] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:56 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:56.388658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7261b830-9709-4a2d-8f0c-10ff20be0955 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:56.389554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7261b830-9709-4a2d-8f0c-10ff20be0955 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:56.393718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-253c5ce5-59b8-45fb-b5c8-753aa57ba2d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:56.394523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-253c5ce5-59b8-45fb-b5c8-753aa57ba2d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:56.396692 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7261b830-9709-4a2d-8f0c-10ff20be0955 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:56.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7261b830-9709-4a2d-8f0c-10ff20be0955 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:56.398061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7261b830-9709-4a2d-8f0c-10ff20be0955 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:56.398541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7261b830-9709-4a2d-8f0c-10ff20be0955 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:56.400842 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-253c5ce5-59b8-45fb-b5c8-753aa57ba2d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:56.401294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-253c5ce5-59b8-45fb-b5c8-753aa57ba2d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:56.402046 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-253c5ce5-59b8-45fb-b5c8-753aa57ba2d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:56.402612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-253c5ce5-59b8-45fb-b5c8-753aa57ba2d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:56.487635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-253c5ce5-59b8-45fb-b5c8-753aa57ba2d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:56.496098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-253c5ce5-59b8-45fb-b5c8-753aa57ba2d1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.106595 May 25 16:40:56.496645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 875/1770] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:56 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:56.554044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7261b830-9709-4a2d-8f0c-10ff20be0955 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:56.562639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7261b830-9709-4a2d-8f0c-10ff20be0955 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.178010 May 25 16:40:56.563282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 896/1771] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:56 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:57.405339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-12b57569-60b8-4539-8deb-a34358085724 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:57.406213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:57.406857 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-12b57569-60b8-4539-8deb-a34358085724 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:57.407028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:57.418193 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:57.418644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:57.418817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-12b57569-60b8-4539-8deb-a34358085724 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:57.419558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-12b57569-60b8-4539-8deb-a34358085724 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:57.421695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:57.421843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-12b57569-60b8-4539-8deb-a34358085724 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:57.421943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:57.422035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-12b57569-60b8-4539-8deb-a34358085724 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:57.620524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:57.623503 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-12b57569-60b8-4539-8deb-a34358085724 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:57.629665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:57.631176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:57.631627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-12b57569-60b8-4539-8deb-a34358085724 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.233969 May 25 16:40:57.632190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 876/1772] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:57 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:57.632350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-408abaea-ca28-4399-9f7c-fa36fe74a706 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.233572 May 25 16:40:57.632919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 897/1773] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:57.636634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-69db4ec1-16c5-4bd5-b65f-11904f426b42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:57.637244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-69db4ec1-16c5-4bd5-b65f-11904f426b42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:57.638191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3e529542-edf9-4477-b9ba-752bd1b3c90f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:57.639326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3e529542-edf9-4477-b9ba-752bd1b3c90f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:57.643777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69db4ec1-16c5-4bd5-b65f-11904f426b42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:57.644237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69db4ec1-16c5-4bd5-b65f-11904f426b42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:57.645003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69db4ec1-16c5-4bd5-b65f-11904f426b42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:57.645440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69db4ec1-16c5-4bd5-b65f-11904f426b42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:57.646433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e529542-edf9-4477-b9ba-752bd1b3c90f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:57.646878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e529542-edf9-4477-b9ba-752bd1b3c90f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:57.647676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e529542-edf9-4477-b9ba-752bd1b3c90f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:57.648105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e529542-edf9-4477-b9ba-752bd1b3c90f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:57.714175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-69db4ec1-16c5-4bd5-b65f-11904f426b42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:57.723316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-69db4ec1-16c5-4bd5-b65f-11904f426b42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090043 May 25 16:40:57.724005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 877/1774] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:57 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:57.805092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3e529542-edf9-4477-b9ba-752bd1b3c90f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:57.813992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3e529542-edf9-4477-b9ba-752bd1b3c90f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.179917 May 25 16:40:57.814567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 898/1775] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:57 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:58.653015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:58.653743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-51b93bae-70d9-4f44-8fba-7daa9f42bd16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:58.654135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:58.654255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-51b93bae-70d9-4f44-8fba-7daa9f42bd16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:58.660636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:58.660825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51b93bae-70d9-4f44-8fba-7daa9f42bd16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:58.661082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:58.661241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51b93bae-70d9-4f44-8fba-7daa9f42bd16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:58.661835 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:58.661983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51b93bae-70d9-4f44-8fba-7daa9f42bd16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:58.662261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:58.662409 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51b93bae-70d9-4f44-8fba-7daa9f42bd16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:58.678897 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:40:58.793670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:40:58.793862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:40:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a (ip 10.222.0.45) !!! May 25 16:40:58.863731 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:58.866693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51b93bae-70d9-4f44-8fba-7daa9f42bd16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:58.872756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:58.874291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:58.875067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-51b93bae-70d9-4f44-8fba-7daa9f42bd16 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.227295 May 25 16:40:58.875459 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9a4debfe-7d27-4c83-bcfe-a378836ce595 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.227720 May 25 16:40:58.875673 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 878/1776] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:58 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:58.876246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 899/1777] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:58 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:40:58.880262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7dcfbd44-4500-4fdb-abd5-9c5f7aa6c250 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:58.881072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7dcfbd44-4500-4fdb-abd5-9c5f7aa6c250 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:58.881490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-85df9dc8-11ba-41c4-b437-102527fa7433 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:58.882200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-85df9dc8-11ba-41c4-b437-102527fa7433 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:58.887626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dcfbd44-4500-4fdb-abd5-9c5f7aa6c250 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:58.888076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dcfbd44-4500-4fdb-abd5-9c5f7aa6c250 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:58.888791 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85df9dc8-11ba-41c4-b437-102527fa7433 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:58.888942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dcfbd44-4500-4fdb-abd5-9c5f7aa6c250 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:58.889254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7dcfbd44-4500-4fdb-abd5-9c5f7aa6c250 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:58.889409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85df9dc8-11ba-41c4-b437-102527fa7433 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:58.890147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85df9dc8-11ba-41c4-b437-102527fa7433 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:58.890586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85df9dc8-11ba-41c4-b437-102527fa7433 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:58.970239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7dcfbd44-4500-4fdb-abd5-9c5f7aa6c250 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:58.978804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7dcfbd44-4500-4fdb-abd5-9c5f7aa6c250 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.102150 May 25 16:40:58.979471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 879/1778] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:58 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:59.046586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-85df9dc8-11ba-41c4-b437-102527fa7433 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:40:59.059506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-85df9dc8-11ba-41c4-b437-102527fa7433 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.181330 May 25 16:40:59.060388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 900/1779] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:58 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:40:59.896568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ddc8b31e-ad70-4d4b-b4d5-33f3b9321728 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:59.897449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:40:59.898008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ddc8b31e-ad70-4d4b-b4d5-33f3b9321728 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:59.898144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:40:59.907728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ddc8b31e-ad70-4d4b-b4d5-33f3b9321728 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:59.907986 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:59.908418 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:59.908657 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ddc8b31e-ad70-4d4b-b4d5-33f3b9321728 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:59.909580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ddc8b31e-ad70-4d4b-b4d5-33f3b9321728 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:59.909802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:40:59.910242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ddc8b31e-ad70-4d4b-b4d5-33f3b9321728 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:40:59.910469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:00.105341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ddc8b31e-ad70-4d4b-b4d5-33f3b9321728 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:00.109418 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:00.113469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ddc8b31e-ad70-4d4b-b4d5-33f3b9321728 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222753 May 25 16:41:00.114032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 880/1780] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:40:59 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:00.118420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:00.118606 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-897e1aa9-7b6a-4202-ac75-b6e283a4523d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:00.119274 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-897e1aa9-7b6a-4202-ac75-b6e283a4523d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:00.120044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:00.121219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2c341310-4e7e-4d11-8c05-b6eb114c0501 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.230478 May 25 16:41:00.121751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 901/1781] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:40:59 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:00.125360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-897e1aa9-7b6a-4202-ac75-b6e283a4523d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:00.125812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-897e1aa9-7b6a-4202-ac75-b6e283a4523d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:00.126566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-897e1aa9-7b6a-4202-ac75-b6e283a4523d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:00.126693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cae1addf-4886-481c-aa69-9be917cef6a2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:00.126995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-897e1aa9-7b6a-4202-ac75-b6e283a4523d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:00.127303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cae1addf-4886-481c-aa69-9be917cef6a2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:00.133254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cae1addf-4886-481c-aa69-9be917cef6a2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:00.134121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cae1addf-4886-481c-aa69-9be917cef6a2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:00.134661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cae1addf-4886-481c-aa69-9be917cef6a2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:00.135295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cae1addf-4886-481c-aa69-9be917cef6a2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:00.194904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-897e1aa9-7b6a-4202-ac75-b6e283a4523d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:00.203536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-897e1aa9-7b6a-4202-ac75-b6e283a4523d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088481 May 25 16:41:00.204298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 881/1782] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:00 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:00.282465 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cae1addf-4886-481c-aa69-9be917cef6a2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:00.294104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cae1addf-4886-481c-aa69-9be917cef6a2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.170972 May 25 16:41:00.294613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 902/1783] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:00 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:01.130013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a217d284-ed7e-437c-80c0-e019d8a5b270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:01.130618 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a217d284-ed7e-437c-80c0-e019d8a5b270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:01.137634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a217d284-ed7e-437c-80c0-e019d8a5b270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:01.138107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a217d284-ed7e-437c-80c0-e019d8a5b270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:01.138858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a217d284-ed7e-437c-80c0-e019d8a5b270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:01.139458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a217d284-ed7e-437c-80c0-e019d8a5b270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:01.140380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:01.140380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:01.146558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:01.147007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:01.147798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:01.148231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:01.328663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a217d284-ed7e-437c-80c0-e019d8a5b270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:01.335695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:01.337104 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a217d284-ed7e-437c-80c0-e019d8a5b270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.210922 May 25 16:41:01.337647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 882/1784] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:01 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:01.342220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aca461fd-81c4-49f0-8708-4f05050a2563 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:01.342817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aca461fd-81c4-49f0-8708-4f05050a2563 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:01.344817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:01.346592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:01.347972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e036fd3d-04be-46ec-9803-e2c5e96abdb7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.212713 May 25 16:41:01.348613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 903/1785] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:01 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:01.351233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aca461fd-81c4-49f0-8708-4f05050a2563 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:01.351233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aca461fd-81c4-49f0-8708-4f05050a2563 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:01.351233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aca461fd-81c4-49f0-8708-4f05050a2563 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:01.351233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aca461fd-81c4-49f0-8708-4f05050a2563 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:01.353299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-be1d36c4-7a31-4aab-9f7b-d3525cd7ac8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:01.354135 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-be1d36c4-7a31-4aab-9f7b-d3525cd7ac8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:01.360241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be1d36c4-7a31-4aab-9f7b-d3525cd7ac8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:01.360694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be1d36c4-7a31-4aab-9f7b-d3525cd7ac8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:01.361445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be1d36c4-7a31-4aab-9f7b-d3525cd7ac8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:01.361877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be1d36c4-7a31-4aab-9f7b-d3525cd7ac8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:01.436941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aca461fd-81c4-49f0-8708-4f05050a2563 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:01.445119 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aca461fd-81c4-49f0-8708-4f05050a2563 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.106333 May 25 16:41:01.445732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 883/1786] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:01 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:01.505233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-be1d36c4-7a31-4aab-9f7b-d3525cd7ac8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:01.517625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-be1d36c4-7a31-4aab-9f7b-d3525cd7ac8b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.167650 May 25 16:41:01.518431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 904/1787] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:01 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:02.359124 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e3977b7b-5a49-445b-9dae-9242299ffd99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:02.360022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e3977b7b-5a49-445b-9dae-9242299ffd99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:02.370053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3977b7b-5a49-445b-9dae-9242299ffd99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:02.370246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:02.370657 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3977b7b-5a49-445b-9dae-9242299ffd99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:02.370860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:02.371271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3977b7b-5a49-445b-9dae-9242299ffd99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:02.371729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3977b7b-5a49-445b-9dae-9242299ffd99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:02.377673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:02.378124 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:02.378882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:02.379364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:02.381962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:02.577310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e3977b7b-5a49-445b-9dae-9242299ffd99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:02.579901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:02.585731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e3977b7b-5a49-445b-9dae-9242299ffd99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.233285 May 25 16:41:02.586276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 884/1788] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:02 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:02.588591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:02.590144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:02.590919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0c02d505-54af-4c7a-b850-b95e322522e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:02.591345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-eeb4d427-e29a-4100-be0e-ceeb4bd3ebbd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.227565 May 25 16:41:02.591550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0c02d505-54af-4c7a-b850-b95e322522e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:02.592070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 905/1789] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:02 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:02.596788 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-283812fe-6add-4ebd-b3b7-eb9f2214e923 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:02.597387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-283812fe-6add-4ebd-b3b7-eb9f2214e923 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:02.597712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0c02d505-54af-4c7a-b850-b95e322522e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:02.598148 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0c02d505-54af-4c7a-b850-b95e322522e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:02.599094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0c02d505-54af-4c7a-b850-b95e322522e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:02.599699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0c02d505-54af-4c7a-b850-b95e322522e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:02.604579 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-283812fe-6add-4ebd-b3b7-eb9f2214e923 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:02.605006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-283812fe-6add-4ebd-b3b7-eb9f2214e923 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:02.605963 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-283812fe-6add-4ebd-b3b7-eb9f2214e923 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:02.606574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-283812fe-6add-4ebd-b3b7-eb9f2214e923 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:02.669297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0c02d505-54af-4c7a-b850-b95e322522e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:02.677634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0c02d505-54af-4c7a-b850-b95e322522e4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090404 May 25 16:41:02.678241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 885/1790] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:02 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:02.752088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-283812fe-6add-4ebd-b3b7-eb9f2214e923 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:02.760247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-283812fe-6add-4ebd-b3b7-eb9f2214e923 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.166994 May 25 16:41:02.760792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 906/1791] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:02 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:03.445640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:41:03.446626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:41:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:41:03.611770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-07a10ac7-603e-4e99-b9cc-a35c287a0a94 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:03.612824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-07a10ac7-603e-4e99-b9cc-a35c287a0a94 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:03.620055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:03.621772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:03.623815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-07a10ac7-603e-4e99-b9cc-a35c287a0a94 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:03.624565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-07a10ac7-603e-4e99-b9cc-a35c287a0a94 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:03.625776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-07a10ac7-603e-4e99-b9cc-a35c287a0a94 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:03.626494 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-07a10ac7-603e-4e99-b9cc-a35c287a0a94 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:03.631637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:03.632366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:03.633610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:03.634319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:03.820560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-07a10ac7-603e-4e99-b9cc-a35c287a0a94 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:03.824041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:03.829462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-07a10ac7-603e-4e99-b9cc-a35c287a0a94 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.224194 May 25 16:41:03.830044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 886/1792] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:03 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:03.833299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:03.834452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9b8257f6-2b56-44ec-86fe-3c69118ccf13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:03.834926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:03.835109 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9b8257f6-2b56-44ec-86fe-3c69118ccf13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:03.836112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-523acaa6-957b-405d-9cdb-f9938e3fdf25 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.222684 May 25 16:41:03.836846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 907/1793] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:03 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:03.841081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2494809d-ef4f-46c0-81cc-b37cab0561cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:03.841684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2494809d-ef4f-46c0-81cc-b37cab0561cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:03.847604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2494809d-ef4f-46c0-81cc-b37cab0561cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:03.848075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2494809d-ef4f-46c0-81cc-b37cab0561cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:03.848830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2494809d-ef4f-46c0-81cc-b37cab0561cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:03.849256 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2494809d-ef4f-46c0-81cc-b37cab0561cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:03.850463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b8257f6-2b56-44ec-86fe-3c69118ccf13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:03.850932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b8257f6-2b56-44ec-86fe-3c69118ccf13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:03.851759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b8257f6-2b56-44ec-86fe-3c69118ccf13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:03.852189 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9b8257f6-2b56-44ec-86fe-3c69118ccf13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:03.945300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9b8257f6-2b56-44ec-86fe-3c69118ccf13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:03.953609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9b8257f6-2b56-44ec-86fe-3c69118ccf13 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.122590 May 25 16:41:03.954194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 887/1794] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:03 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:04.008628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2494809d-ef4f-46c0-81cc-b37cab0561cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:04.020409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2494809d-ef4f-46c0-81cc-b37cab0561cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.182590 May 25 16:41:04.021216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 908/1795] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:03 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:04.847082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ef0b7967-4fa8-4f12-a745-e0738db299e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:04.847727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ef0b7967-4fa8-4f12-a745-e0738db299e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:04.854030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:04.854658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:04.855056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef0b7967-4fa8-4f12-a745-e0738db299e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:04.855543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef0b7967-4fa8-4f12-a745-e0738db299e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:04.856315 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef0b7967-4fa8-4f12-a745-e0738db299e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:04.856752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ef0b7967-4fa8-4f12-a745-e0738db299e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:04.861689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:04.862136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:04.862890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:04.863349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:05.049239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ef0b7967-4fa8-4f12-a745-e0738db299e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:05.053224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:05.057667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ef0b7967-4fa8-4f12-a745-e0738db299e4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.214477 May 25 16:41:05.058328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 888/1796] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:04 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:05.062106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:05.062821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e1b9e978-e3d3-4a13-8cdd-f862569298d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:05.066045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e1b9e978-e3d3-4a13-8cdd-f862569298d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:05.066204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:05.066204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d7dff405-ca26-4f42-8a11-ce4abb8af922 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.215023 May 25 16:41:05.066204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 909/1797] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:04 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:05.070731 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3784e8c3-5784-4be8-8ed0-86fd93cb8265 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:05.071347 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3784e8c3-5784-4be8-8ed0-86fd93cb8265 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:05.077808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3784e8c3-5784-4be8-8ed0-86fd93cb8265 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:05.078259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3784e8c3-5784-4be8-8ed0-86fd93cb8265 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:05.079017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3784e8c3-5784-4be8-8ed0-86fd93cb8265 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:05.079522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3784e8c3-5784-4be8-8ed0-86fd93cb8265 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:05.080114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e1b9e978-e3d3-4a13-8cdd-f862569298d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:05.080634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e1b9e978-e3d3-4a13-8cdd-f862569298d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:05.081470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e1b9e978-e3d3-4a13-8cdd-f862569298d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:05.081974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e1b9e978-e3d3-4a13-8cdd-f862569298d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:05.174354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e1b9e978-e3d3-4a13-8cdd-f862569298d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:05.182715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e1b9e978-e3d3-4a13-8cdd-f862569298d7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.123433 May 25 16:41:05.183306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 889/1798] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:05 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:05.242423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3784e8c3-5784-4be8-8ed0-86fd93cb8265 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:05.251577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3784e8c3-5784-4be8-8ed0-86fd93cb8265 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.184253 May 25 16:41:05.252130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 910/1799] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:05 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:06.079196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8bd74864-b12f-4530-a716-dc3a7d36e661 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:06.080251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8bd74864-b12f-4530-a716-dc3a7d36e661 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:06.087112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:06.088396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:06.090823 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bd74864-b12f-4530-a716-dc3a7d36e661 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:06.091457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bd74864-b12f-4530-a716-dc3a7d36e661 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:06.092234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bd74864-b12f-4530-a716-dc3a7d36e661 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:06.092665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8bd74864-b12f-4530-a716-dc3a7d36e661 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:06.095262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:06.095705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:06.096456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:06.096883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:06.290006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8bd74864-b12f-4530-a716-dc3a7d36e661 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:06.292926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:06.298615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8bd74864-b12f-4530-a716-dc3a7d36e661 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.225769 May 25 16:41:06.299536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 890/1800] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:06 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:06.303941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1d290613-fea1-4807-a8d3-880c34e393fc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:06.304566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1d290613-fea1-4807-a8d3-880c34e393fc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:06.310560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1d290613-fea1-4807-a8d3-880c34e393fc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:06.311010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1d290613-fea1-4807-a8d3-880c34e393fc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:06.311604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:06.311822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1d290613-fea1-4807-a8d3-880c34e393fc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:06.312222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1d290613-fea1-4807-a8d3-880c34e393fc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:06.313123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:06.314308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b9010221-a856-4159-8990-6f2908d4e04b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.233703 May 25 16:41:06.314867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 911/1801] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:06 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:06.319213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5d45f3c2-c689-4314-9f57-02189cc71fdc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:06.319817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5d45f3c2-c689-4314-9f57-02189cc71fdc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:06.325848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d45f3c2-c689-4314-9f57-02189cc71fdc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:06.326233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d45f3c2-c689-4314-9f57-02189cc71fdc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:06.327282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d45f3c2-c689-4314-9f57-02189cc71fdc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:06.327511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d45f3c2-c689-4314-9f57-02189cc71fdc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:06.375390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1d290613-fea1-4807-a8d3-880c34e393fc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:06.384072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1d290613-fea1-4807-a8d3-880c34e393fc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083542 May 25 16:41:06.384619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 891/1802] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:06 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:06.469640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5d45f3c2-c689-4314-9f57-02189cc71fdc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:06.477757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5d45f3c2-c689-4314-9f57-02189cc71fdc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.161903 May 25 16:41:06.478320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 912/1803] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:06 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:07.320734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d380ff27-6a15-43af-93a5-df8bbf25f696 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:07.321747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d380ff27-6a15-43af-93a5-df8bbf25f696 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:07.329356 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d380ff27-6a15-43af-93a5-df8bbf25f696 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:07.329804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d380ff27-6a15-43af-93a5-df8bbf25f696 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:07.330656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d380ff27-6a15-43af-93a5-df8bbf25f696 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:07.331090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d380ff27-6a15-43af-93a5-df8bbf25f696 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:07.332552 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:07.333198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:07.340461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:07.340919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:07.341702 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:07.342133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:07.529852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d380ff27-6a15-43af-93a5-df8bbf25f696 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:07.532446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:07.538925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d380ff27-6a15-43af-93a5-df8bbf25f696 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.225204 May 25 16:41:07.539977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 892/1804] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:07 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:07.541542 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:07.543081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:07.544272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-496d2588-a9d9-4a9b-8f9f-bbe229f35a9f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.215688 May 25 16:41:07.544595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b39433cc-8982-4fe8-9c92-fc3f1975a5ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:07.545083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 913/1805] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:07 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:07.545244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b39433cc-8982-4fe8-9c92-fc3f1975a5ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:07.551502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cd158d71-79b0-442f-8a34-96852893e974 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:07.551502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cd158d71-79b0-442f-8a34-96852893e974 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:07.551745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b39433cc-8982-4fe8-9c92-fc3f1975a5ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:07.551854 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b39433cc-8982-4fe8-9c92-fc3f1975a5ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:07.552660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b39433cc-8982-4fe8-9c92-fc3f1975a5ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:07.553145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b39433cc-8982-4fe8-9c92-fc3f1975a5ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:07.557034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd158d71-79b0-442f-8a34-96852893e974 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:07.557502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd158d71-79b0-442f-8a34-96852893e974 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:07.558275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd158d71-79b0-442f-8a34-96852893e974 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:07.558708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cd158d71-79b0-442f-8a34-96852893e974 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:07.622163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b39433cc-8982-4fe8-9c92-fc3f1975a5ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:07.630281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b39433cc-8982-4fe8-9c92-fc3f1975a5ca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089348 May 25 16:41:07.630855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 893/1806] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:07 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:07.711283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cd158d71-79b0-442f-8a34-96852893e974 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:07.721276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cd158d71-79b0-442f-8a34-96852893e974 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.175166 May 25 16:41:07.722225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 914/1807] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:07 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:08.561264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-edef06ea-4bef-49b8-8a79-1fa0be91e3da tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:08.561878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-edef06ea-4bef-49b8-8a79-1fa0be91e3da tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:08.563694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:08.564334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:08.569419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-edef06ea-4bef-49b8-8a79-1fa0be91e3da tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:08.569908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-edef06ea-4bef-49b8-8a79-1fa0be91e3da tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:08.570705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-edef06ea-4bef-49b8-8a79-1fa0be91e3da tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:08.571179 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-edef06ea-4bef-49b8-8a79-1fa0be91e3da tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:08.571388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:08.571868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:08.572657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:08.573104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:08.761255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:08.764302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-edef06ea-4bef-49b8-8a79-1fa0be91e3da tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:08.770793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:08.772360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:08.772650 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-edef06ea-4bef-49b8-8a79-1fa0be91e3da tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.215846 May 25 16:41:08.773221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 894/1808] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:08 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:08.773536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3b49f3b9-13a3-41a2-af6e-7279822923fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.213961 May 25 16:41:08.774064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 915/1809] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:08 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:08.778618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-aca9bda1-4015-46b5-b2f9-1b89669af3bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:08.779289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-aca9bda1-4015-46b5-b2f9-1b89669af3bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:08.784379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2a6a4bee-b2f0-4f91-b56c-dbe56439dbfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:08.785351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2a6a4bee-b2f0-4f91-b56c-dbe56439dbfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:08.786650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aca9bda1-4015-46b5-b2f9-1b89669af3bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:08.787341 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aca9bda1-4015-46b5-b2f9-1b89669af3bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:08.788397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aca9bda1-4015-46b5-b2f9-1b89669af3bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:08.789061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aca9bda1-4015-46b5-b2f9-1b89669af3bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:08.793603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a6a4bee-b2f0-4f91-b56c-dbe56439dbfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:08.794059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a6a4bee-b2f0-4f91-b56c-dbe56439dbfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:08.794820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a6a4bee-b2f0-4f91-b56c-dbe56439dbfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:08.795291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a6a4bee-b2f0-4f91-b56c-dbe56439dbfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:08.800250 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:08.877555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2a6a4bee-b2f0-4f91-b56c-dbe56439dbfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:08.888566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2a6a4bee-b2f0-4f91-b56c-dbe56439dbfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.108215 May 25 16:41:08.889134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 895/1810] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:08 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:08.945206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aca9bda1-4015-46b5-b2f9-1b89669af3bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:08.954061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-aca9bda1-4015-46b5-b2f9-1b89669af3bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.178970 May 25 16:41:08.954636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 916/1811] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:08 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:09.794182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b75d81f0-5ca8-4502-86c1-164857d41ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:09.795131 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b75d81f0-5ca8-4502-86c1-164857d41ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:09.795908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:09.797083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:09.805444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b75d81f0-5ca8-4502-86c1-164857d41ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:09.806215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b75d81f0-5ca8-4502-86c1-164857d41ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:09.807371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b75d81f0-5ca8-4502-86c1-164857d41ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:09.807523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:09.807804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b75d81f0-5ca8-4502-86c1-164857d41ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:09.807984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:09.808748 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:09.809174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:09.998463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b75d81f0-5ca8-4502-86c1-164857d41ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:10.001043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:10.007040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b75d81f0-5ca8-4502-86c1-164857d41ca3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.219639 May 25 16:41:10.017304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 896/1812] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:09 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:10.017508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:10.017508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:10.017734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9282de33-e305-4ea1-9893-00926359699b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:10.017734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9282de33-e305-4ea1-9893-00926359699b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:10.017989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-09ad1bb1-f6fe-4472-9ad5-20c105d6a79c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.224166 May 25 16:41:10.017989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 917/1813] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:09 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:10.017989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-96147de5-fad3-46d1-ad6e-b723678d1e99 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:10.018378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-96147de5-fad3-46d1-ad6e-b723678d1e99 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:10.018718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9282de33-e305-4ea1-9893-00926359699b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:10.019257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9282de33-e305-4ea1-9893-00926359699b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:10.020053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9282de33-e305-4ea1-9893-00926359699b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:10.020485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9282de33-e305-4ea1-9893-00926359699b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:10.024300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-96147de5-fad3-46d1-ad6e-b723678d1e99 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:10.024722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-96147de5-fad3-46d1-ad6e-b723678d1e99 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:10.025618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-96147de5-fad3-46d1-ad6e-b723678d1e99 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:10.026070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-96147de5-fad3-46d1-ad6e-b723678d1e99 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:10.105797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9282de33-e305-4ea1-9893-00926359699b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:10.114177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9282de33-e305-4ea1-9893-00926359699b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.105586 May 25 16:41:10.114966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 897/1814] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:10 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:10.175691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-96147de5-fad3-46d1-ad6e-b723678d1e99 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:10.184411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-96147de5-fad3-46d1-ad6e-b723678d1e99 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.169888 May 25 16:41:10.184774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 918/1815] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:10 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:11.028713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b2cc5e1f-acce-42b6-9d56-f7b00a819504 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:11.029642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b2cc5e1f-acce-42b6-9d56-f7b00a819504 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:11.033268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:11.033583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:11.034613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:11.042347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2cc5e1f-acce-42b6-9d56-f7b00a819504 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:11.042914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2cc5e1f-acce-42b6-9d56-f7b00a819504 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:11.043701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2cc5e1f-acce-42b6-9d56-f7b00a819504 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:11.044093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:11.044206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2cc5e1f-acce-42b6-9d56-f7b00a819504 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:11.044583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:11.045350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:11.045787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:11.234326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b2cc5e1f-acce-42b6-9d56-f7b00a819504 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:11.236947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:11.242367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b2cc5e1f-acce-42b6-9d56-f7b00a819504 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220263 May 25 16:41:11.242946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 898/1816] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:11 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:11.245619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:11.247118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:11.247578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0af9005a-1cc0-42ca-bc09-7fcddf7a067a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:11.248173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0af9005a-1cc0-42ca-bc09-7fcddf7a067a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:11.248300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-38252090-63e6-40f6-b320-78618d688676 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.221097 May 25 16:41:11.258086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 919/1817] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:11 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:11.258086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-13864e9c-d856-4a53-b596-d66f226222dd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:11.258086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-13864e9c-d856-4a53-b596-d66f226222dd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:11.258437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0af9005a-1cc0-42ca-bc09-7fcddf7a067a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:11.258437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0af9005a-1cc0-42ca-bc09-7fcddf7a067a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:11.258437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0af9005a-1cc0-42ca-bc09-7fcddf7a067a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:11.258437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0af9005a-1cc0-42ca-bc09-7fcddf7a067a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:11.259480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13864e9c-d856-4a53-b596-d66f226222dd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:11.259963 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13864e9c-d856-4a53-b596-d66f226222dd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:11.260721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13864e9c-d856-4a53-b596-d66f226222dd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:11.261159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13864e9c-d856-4a53-b596-d66f226222dd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:11.326463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0af9005a-1cc0-42ca-bc09-7fcddf7a067a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:11.334760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0af9005a-1cc0-42ca-bc09-7fcddf7a067a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090814 May 25 16:41:11.335341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 899/1818] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:11 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:11.415207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-13864e9c-d856-4a53-b596-d66f226222dd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:11.428800 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-13864e9c-d856-4a53-b596-d66f226222dd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.178564 May 25 16:41:11.429709 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 920/1819] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:11 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:12.261612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c7decbda-f7ee-48bd-8856-65805b654c61 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:12.262225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c7decbda-f7ee-48bd-8856-65805b654c61 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:12.266247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:12.266955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:12.268868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7decbda-f7ee-48bd-8856-65805b654c61 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:12.269305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7decbda-f7ee-48bd-8856-65805b654c61 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:12.270058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7decbda-f7ee-48bd-8856-65805b654c61 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:12.270475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7decbda-f7ee-48bd-8856-65805b654c61 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:12.274133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:12.274653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:12.275644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:12.276177 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:12.463575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c7decbda-f7ee-48bd-8856-65805b654c61 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:12.466745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:12.469225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:12.478140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:12.480126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:12.480667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c7decbda-f7ee-48bd-8856-65805b654c61 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.224406 May 25 16:41:12.481545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 900/1820] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:12 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:12.481686 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-70cb556d-2d68-45ef-989b-d89621800c1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.219397 May 25 16:41:12.492171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 921/1821] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:12 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:12.492171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-639cd5c3-006f-42a2-88e2-550178a0300e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:12.492171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-639cd5c3-006f-42a2-88e2-550178a0300e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:12.494280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-639cd5c3-006f-42a2-88e2-550178a0300e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:12.494803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-639cd5c3-006f-42a2-88e2-550178a0300e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:12.495755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-639cd5c3-006f-42a2-88e2-550178a0300e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:12.496212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-639cd5c3-006f-42a2-88e2-550178a0300e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:12.497864 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-84100f10-2d37-429e-a7cf-bfa133f21d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:12.498488 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-84100f10-2d37-429e-a7cf-bfa133f21d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:12.505169 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-84100f10-2d37-429e-a7cf-bfa133f21d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:12.505656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-84100f10-2d37-429e-a7cf-bfa133f21d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:12.506477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-84100f10-2d37-429e-a7cf-bfa133f21d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:12.506930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-84100f10-2d37-429e-a7cf-bfa133f21d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:12.589615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-84100f10-2d37-429e-a7cf-bfa133f21d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:12.597755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-84100f10-2d37-429e-a7cf-bfa133f21d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.105085 May 25 16:41:12.598398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 901/1822] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:12 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:12.654297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-639cd5c3-006f-42a2-88e2-550178a0300e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:12.667337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-639cd5c3-006f-42a2-88e2-550178a0300e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.183882 May 25 16:41:12.667910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 922/1823] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:12 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:13.507933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:13.508772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1864acef-e284-417c-91fb-0e55d3fbf327 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:13.509412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:13.509585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1864acef-e284-417c-91fb-0e55d3fbf327 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:13.521466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1864acef-e284-417c-91fb-0e55d3fbf327 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:13.525617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1864acef-e284-417c-91fb-0e55d3fbf327 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:13.525617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1864acef-e284-417c-91fb-0e55d3fbf327 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:13.525617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1864acef-e284-417c-91fb-0e55d3fbf327 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:13.526432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:13.526432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:13.526773 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:13.527240 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:13.966320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1864acef-e284-417c-91fb-0e55d3fbf327 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:13.970921 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:13.977592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1864acef-e284-417c-91fb-0e55d3fbf327 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.475810 May 25 16:41:13.978305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 923/1824] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:13 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:41:13.981598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:13.983047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a3261df9-0348-49fe-b2a6-54166559cf6d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:13.983206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:13.983678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a3261df9-0348-49fe-b2a6-54166559cf6d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:13.984316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-963751b8-27de-443f-9dc7-5af617c2e6f7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.486253 May 25 16:41:13.988100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 902/1825] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:13 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:13.989327 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-00c6fafe-decc-4ad3-8864-7510ac3195db tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:13.989683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a3261df9-0348-49fe-b2a6-54166559cf6d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:13.989894 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-00c6fafe-decc-4ad3-8864-7510ac3195db tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:13.990167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a3261df9-0348-49fe-b2a6-54166559cf6d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:13.990891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a3261df9-0348-49fe-b2a6-54166559cf6d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:13.991563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a3261df9-0348-49fe-b2a6-54166559cf6d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:13.996789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00c6fafe-decc-4ad3-8864-7510ac3195db tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:13.997474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00c6fafe-decc-4ad3-8864-7510ac3195db tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:13.998557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00c6fafe-decc-4ad3-8864-7510ac3195db tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:13.998993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00c6fafe-decc-4ad3-8864-7510ac3195db tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:14.087019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a3261df9-0348-49fe-b2a6-54166559cf6d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:14.096367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a3261df9-0348-49fe-b2a6-54166559cf6d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.117066 May 25 16:41:14.097027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 924/1826] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:13 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:14.158104 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-00c6fafe-decc-4ad3-8864-7510ac3195db tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:14.169558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-00c6fafe-decc-4ad3-8864-7510ac3195db tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.183766 May 25 16:41:14.170130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 903/1827] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:13 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:14.998183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4c9726e0-0fe6-4d1b-af67-689edd416101 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:14.998876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4c9726e0-0fe6-4d1b-af67-689edd416101 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:15.002983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:15.003725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:15.006797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c9726e0-0fe6-4d1b-af67-689edd416101 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:15.007368 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c9726e0-0fe6-4d1b-af67-689edd416101 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:15.008253 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c9726e0-0fe6-4d1b-af67-689edd416101 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:15.008759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c9726e0-0fe6-4d1b-af67-689edd416101 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:15.011587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:15.012116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:15.014579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:15.014579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:15.206137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4c9726e0-0fe6-4d1b-af67-689edd416101 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:15.210204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:15.214859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4c9726e0-0fe6-4d1b-af67-689edd416101 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222355 May 25 16:41:15.215472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 925/1828] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:14 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:15.219610 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:15.219832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-608301c2-fbec-4d98-96e7-4176cc2568f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:15.220435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-608301c2-fbec-4d98-96e7-4176cc2568f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:15.221357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:15.222887 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ea1d984e-93ff-4514-aa5e-94908262ce32 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.224271 May 25 16:41:15.223526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 904/1829] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:14 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:15.226674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-608301c2-fbec-4d98-96e7-4176cc2568f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:15.227129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-608301c2-fbec-4d98-96e7-4176cc2568f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:15.227940 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-608301c2-fbec-4d98-96e7-4176cc2568f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:15.228106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c917ff39-f3bf-416a-bc52-8f2086bec5f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:15.228357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-608301c2-fbec-4d98-96e7-4176cc2568f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:15.228651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c917ff39-f3bf-416a-bc52-8f2086bec5f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:15.235031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c917ff39-f3bf-416a-bc52-8f2086bec5f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:15.235543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c917ff39-f3bf-416a-bc52-8f2086bec5f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:15.236289 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c917ff39-f3bf-416a-bc52-8f2086bec5f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:15.236728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c917ff39-f3bf-416a-bc52-8f2086bec5f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:15.299828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-608301c2-fbec-4d98-96e7-4176cc2568f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:15.308134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-608301c2-fbec-4d98-96e7-4176cc2568f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091707 May 25 16:41:15.308707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 926/1830] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:15 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:15.382588 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c917ff39-f3bf-416a-bc52-8f2086bec5f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:15.390742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c917ff39-f3bf-416a-bc52-8f2086bec5f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.166189 May 25 16:41:15.391330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 905/1831] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:15 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:16.233478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f8872e24-d3e8-4784-9b99-7cfcf7c405db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:16.234193 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f8872e24-d3e8-4784-9b99-7cfcf7c405db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:16.241161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f8872e24-d3e8-4784-9b99-7cfcf7c405db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:16.241366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:16.241818 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f8872e24-d3e8-4784-9b99-7cfcf7c405db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:16.241929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:16.242354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f8872e24-d3e8-4784-9b99-7cfcf7c405db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:16.242782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f8872e24-d3e8-4784-9b99-7cfcf7c405db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:16.249359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:16.249822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:16.250576 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:16.251005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:16.448479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f8872e24-d3e8-4784-9b99-7cfcf7c405db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:16.451107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:16.456554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f8872e24-d3e8-4784-9b99-7cfcf7c405db tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.227098 May 25 16:41:16.457109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 927/1832] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:16 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:16.460014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:16.461499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-94993174-9670-4e7e-bdb6-e7d64c232865 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:16.461644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:16.462108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-94993174-9670-4e7e-bdb6-e7d64c232865 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:16.462660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8343f9e6-bf6b-4af4-bf73-0c4047bf77ee tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.225374 May 25 16:41:16.463554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 906/1833] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:16 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:16.468159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94993174-9670-4e7e-bdb6-e7d64c232865 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:16.468613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94993174-9670-4e7e-bdb6-e7d64c232865 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:16.469373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94993174-9670-4e7e-bdb6-e7d64c232865 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:16.469821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94993174-9670-4e7e-bdb6-e7d64c232865 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:16.471028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-83745e41-5d79-4bf8-b67d-713fcc2d28fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:16.471655 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-83745e41-5d79-4bf8-b67d-713fcc2d28fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:16.477682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83745e41-5d79-4bf8-b67d-713fcc2d28fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:16.478793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83745e41-5d79-4bf8-b67d-713fcc2d28fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:16.479641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83745e41-5d79-4bf8-b67d-713fcc2d28fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:16.480208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83745e41-5d79-4bf8-b67d-713fcc2d28fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:16.559366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-94993174-9670-4e7e-bdb6-e7d64c232865 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:16.567747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-94993174-9670-4e7e-bdb6-e7d64c232865 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.109640 May 25 16:41:16.568561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 928/1834] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:16 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:16.628817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-83745e41-5d79-4bf8-b67d-713fcc2d28fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:16.641262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-83745e41-5d79-4bf8-b67d-713fcc2d28fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173459 May 25 16:41:16.642022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 907/1835] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:16 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:17.478108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3037906e-6472-496d-907c-23b395bcba85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:17.479105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3037906e-6472-496d-907c-23b395bcba85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:17.483689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=96 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:17.486098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:17.487386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:17.493462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3037906e-6472-496d-907c-23b395bcba85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:17.494196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3037906e-6472-496d-907c-23b395bcba85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:17.495621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3037906e-6472-496d-907c-23b395bcba85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:17.496053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3037906e-6472-496d-907c-23b395bcba85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:17.497968 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:17.498415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:17.499222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:17.499651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:17.698945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3037906e-6472-496d-907c-23b395bcba85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:17.703873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:17.712154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3037906e-6472-496d-907c-23b395bcba85 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.240159 May 25 16:41:17.713222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 929/1836] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:17 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:17.718656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:17.720941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ee086659-ca93-4b99-abe0-1255fbba4eef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:17.721189 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:17.722267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ee086659-ca93-4b99-abe0-1255fbba4eef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:17.723129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-333834ea-9604-476f-9dd0-7769750cdafa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.244628 May 25 16:41:17.728284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 908/1837] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:17 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:17.731052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-80cea6ce-c5d7-4bb3-941a-1ea8fe201b58 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:17.731688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-80cea6ce-c5d7-4bb3-941a-1ea8fe201b58 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:17.732498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee086659-ca93-4b99-abe0-1255fbba4eef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:17.733205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee086659-ca93-4b99-abe0-1255fbba4eef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:17.733859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee086659-ca93-4b99-abe0-1255fbba4eef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:17.734329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee086659-ca93-4b99-abe0-1255fbba4eef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:17.737710 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80cea6ce-c5d7-4bb3-941a-1ea8fe201b58 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:17.738219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80cea6ce-c5d7-4bb3-941a-1ea8fe201b58 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:17.738956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80cea6ce-c5d7-4bb3-941a-1ea8fe201b58 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:17.739457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80cea6ce-c5d7-4bb3-941a-1ea8fe201b58 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:17.817751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ee086659-ca93-4b99-abe0-1255fbba4eef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:17.826111 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ee086659-ca93-4b99-abe0-1255fbba4eef tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.111461 May 25 16:41:17.826683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 930/1838] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:17 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:17.893751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-80cea6ce-c5d7-4bb3-941a-1ea8fe201b58 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:17.902130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-80cea6ce-c5d7-4bb3-941a-1ea8fe201b58 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.176217 May 25 16:41:17.902698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 909/1839] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:17 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:18.740651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cc6bac24-403b-4d15-887b-fdb9a7b0a672 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:18.741502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cc6bac24-403b-4d15-887b-fdb9a7b0a672 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:18.748335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:18.749381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:18.752055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc6bac24-403b-4d15-887b-fdb9a7b0a672 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:18.752795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc6bac24-403b-4d15-887b-fdb9a7b0a672 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:18.754039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc6bac24-403b-4d15-887b-fdb9a7b0a672 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:18.754717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc6bac24-403b-4d15-887b-fdb9a7b0a672 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:18.760007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:18.760883 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:18.761704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:18.762133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:18.952704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cc6bac24-403b-4d15-887b-fdb9a7b0a672 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:18.955884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:18.961177 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cc6bac24-403b-4d15-887b-fdb9a7b0a672 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.227345 May 25 16:41:18.971018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 931/1840] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:18 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:18.971253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:18.971253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:18.971481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d29e16f2-e825-4e2a-b245-c84aecafcd23 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:18.971481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d29e16f2-e825-4e2a-b245-c84aecafcd23 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:18.971669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-50c88bb8-a862-4bc8-bbc0-fcdbeb35af5f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.225672 May 25 16:41:18.971669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 910/1841] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:18.973210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d29e16f2-e825-4e2a-b245-c84aecafcd23 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:18.973368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aa6765c7-0b00-4399-85aa-7d3a82cb0c57 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:18.973637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d29e16f2-e825-4e2a-b245-c84aecafcd23 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:18.973868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aa6765c7-0b00-4399-85aa-7d3a82cb0c57 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:18.974387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d29e16f2-e825-4e2a-b245-c84aecafcd23 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:18.974806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d29e16f2-e825-4e2a-b245-c84aecafcd23 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:18.980135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa6765c7-0b00-4399-85aa-7d3a82cb0c57 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:18.980594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa6765c7-0b00-4399-85aa-7d3a82cb0c57 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:18.981320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa6765c7-0b00-4399-85aa-7d3a82cb0c57 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:18.981747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa6765c7-0b00-4399-85aa-7d3a82cb0c57 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:19.067526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d29e16f2-e825-4e2a-b245-c84aecafcd23 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:19.075465 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d29e16f2-e825-4e2a-b245-c84aecafcd23 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.112671 May 25 16:41:19.076113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 932/1842] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:18 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:19.137454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa6765c7-0b00-4399-85aa-7d3a82cb0c57 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:19.139759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:19.146795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aa6765c7-0b00-4399-85aa-7d3a82cb0c57 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.177577 May 25 16:41:19.147371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 911/1843] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:18 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:19.981627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-391a159b-ca9d-4041-aae5-ce1c708d2002 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:19.982260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-391a159b-ca9d-4041-aae5-ce1c708d2002 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:19.986263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:19.986986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:19.989609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-391a159b-ca9d-4041-aae5-ce1c708d2002 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:19.990097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-391a159b-ca9d-4041-aae5-ce1c708d2002 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:19.990944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-391a159b-ca9d-4041-aae5-ce1c708d2002 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:19.991444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-391a159b-ca9d-4041-aae5-ce1c708d2002 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:19.994352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:19.994837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:19.995670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:19.996129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:20.188235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-391a159b-ca9d-4041-aae5-ce1c708d2002 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:20.192999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:20.196610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-391a159b-ca9d-4041-aae5-ce1c708d2002 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221112 May 25 16:41:20.197162 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 933/1844] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:19 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:20.201922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9b5ad65f-03bb-4968-8044-96012ee7679a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:20.202187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:20.202536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9b5ad65f-03bb-4968-8044-96012ee7679a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:20.203684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:20.205303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f5012e11-dda0-41ae-a4c5-6e58f8f8c864 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.222887 May 25 16:41:20.207995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 912/1845] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:19 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:20.209724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b5ad65f-03bb-4968-8044-96012ee7679a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:20.210154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dc63721e-b6f0-4a53-aeb1-8677f9df77e4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:20.210309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b5ad65f-03bb-4968-8044-96012ee7679a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:20.210762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dc63721e-b6f0-4a53-aeb1-8677f9df77e4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:20.210906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b5ad65f-03bb-4968-8044-96012ee7679a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:20.211403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b5ad65f-03bb-4968-8044-96012ee7679a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:20.217330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc63721e-b6f0-4a53-aeb1-8677f9df77e4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:20.217783 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc63721e-b6f0-4a53-aeb1-8677f9df77e4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:20.218548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc63721e-b6f0-4a53-aeb1-8677f9df77e4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:20.218981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc63721e-b6f0-4a53-aeb1-8677f9df77e4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:20.280859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9b5ad65f-03bb-4968-8044-96012ee7679a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:20.289942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9b5ad65f-03bb-4968-8044-96012ee7679a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091267 May 25 16:41:20.290553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 934/1846] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:20 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:20.377531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dc63721e-b6f0-4a53-aeb1-8677f9df77e4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:20.387813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dc63721e-b6f0-4a53-aeb1-8677f9df77e4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.180949 May 25 16:41:20.388437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 913/1847] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:20 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:21.217495 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cdf8d908-4645-45a2-8043-2bab884b3bed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:21.218068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cdf8d908-4645-45a2-8043-2bab884b3bed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:21.223346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:21.223917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:21.224872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cdf8d908-4645-45a2-8043-2bab884b3bed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:21.225315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cdf8d908-4645-45a2-8043-2bab884b3bed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:21.226070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cdf8d908-4645-45a2-8043-2bab884b3bed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:21.226498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cdf8d908-4645-45a2-8043-2bab884b3bed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:21.230361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:21.230810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:21.231593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:21.232021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:21.423541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cdf8d908-4645-45a2-8043-2bab884b3bed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:21.426431 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:21.432884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cdf8d908-4645-45a2-8043-2bab884b3bed tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220162 May 25 16:41:21.433834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 935/1848] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:21 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:21.435747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:21.437243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:21.438161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3139cb21-401b-4c57-ba70-c055542fa98a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:21.438400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0b23232e-92f5-44aa-9f42-57d63f2b5234 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.218969 May 25 16:41:21.438764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3139cb21-401b-4c57-ba70-c055542fa98a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:21.443502 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 914/1849] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:21 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:21.444382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6feb4d79-7628-49e7-ba75-558608e8c768 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:21.445009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6feb4d79-7628-49e7-ba75-558608e8c768 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:21.445325 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3139cb21-401b-4c57-ba70-c055542fa98a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:21.445782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3139cb21-401b-4c57-ba70-c055542fa98a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:21.446495 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3139cb21-401b-4c57-ba70-c055542fa98a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:21.446916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3139cb21-401b-4c57-ba70-c055542fa98a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:21.451259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6feb4d79-7628-49e7-ba75-558608e8c768 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:21.451709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6feb4d79-7628-49e7-ba75-558608e8c768 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:21.452465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6feb4d79-7628-49e7-ba75-558608e8c768 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:21.452890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6feb4d79-7628-49e7-ba75-558608e8c768 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:21.530611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3139cb21-401b-4c57-ba70-c055542fa98a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:21.538767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3139cb21-401b-4c57-ba70-c055542fa98a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.104155 May 25 16:41:21.539674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 936/1850] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:21 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:21.600140 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6feb4d79-7628-49e7-ba75-558608e8c768 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:21.610119 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6feb4d79-7628-49e7-ba75-558608e8c768 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.169136 May 25 16:41:21.610692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 915/1851] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:21 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:22.452888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3cf4118a-d840-48fd-a279-3c5bfc29f22f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:22.453705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3cf4118a-d840-48fd-a279-3c5bfc29f22f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:22.458645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:22.459393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:22.464670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cf4118a-d840-48fd-a279-3c5bfc29f22f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:22.465527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cf4118a-d840-48fd-a279-3c5bfc29f22f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:22.466466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cf4118a-d840-48fd-a279-3c5bfc29f22f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:22.466943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:22.467096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cf4118a-d840-48fd-a279-3c5bfc29f22f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:22.467514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:22.468404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:22.468914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:22.671097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3cf4118a-d840-48fd-a279-3c5bfc29f22f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:22.674035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:22.679695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3cf4118a-d840-48fd-a279-3c5bfc29f22f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.232619 May 25 16:41:22.680281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 937/1852] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:22 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:22.682778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:22.684431 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:22.684806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-54ed267d-4b51-4506-8646-3a38a19178e2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:22.685413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-54ed267d-4b51-4506-8646-3a38a19178e2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:22.685573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-86224b8f-0414-439a-99fa-438da8fcbc29 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.231882 May 25 16:41:22.691959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 916/1853] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:22 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:22.692113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-54ed267d-4b51-4506-8646-3a38a19178e2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:22.692113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-54ed267d-4b51-4506-8646-3a38a19178e2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:22.692625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-54ed267d-4b51-4506-8646-3a38a19178e2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:22.693055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-54ed267d-4b51-4506-8646-3a38a19178e2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:22.695306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6a950332-2a63-47be-b1a0-5e1696c59920 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:22.695901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6a950332-2a63-47be-b1a0-5e1696c59920 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:22.703200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a950332-2a63-47be-b1a0-5e1696c59920 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:22.703936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a950332-2a63-47be-b1a0-5e1696c59920 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:22.704645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a950332-2a63-47be-b1a0-5e1696c59920 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:22.705076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a950332-2a63-47be-b1a0-5e1696c59920 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:22.717804 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:22.773255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-54ed267d-4b51-4506-8646-3a38a19178e2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:22.781601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-54ed267d-4b51-4506-8646-3a38a19178e2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.100313 May 25 16:41:22.782144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 938/1854] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:22 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:22.863297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6a950332-2a63-47be-b1a0-5e1696c59920 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:22.871871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6a950332-2a63-47be-b1a0-5e1696c59920 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.184828 May 25 16:41:22.872404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 917/1855] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:22 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:23.700405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7323a70d-72ec-4c0a-a266-89f7e31d7cbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:23.701255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7323a70d-72ec-4c0a-a266-89f7e31d7cbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:23.705686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:23.706499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:23.710100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7323a70d-72ec-4c0a-a266-89f7e31d7cbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:23.710722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7323a70d-72ec-4c0a-a266-89f7e31d7cbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:23.711744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7323a70d-72ec-4c0a-a266-89f7e31d7cbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:23.712312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7323a70d-72ec-4c0a-a266-89f7e31d7cbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:23.715132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:23.715801 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:23.716813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:23.719696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:23.916062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:23.919673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7323a70d-72ec-4c0a-a266-89f7e31d7cbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:23.925249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:23.927620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:23.928047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7323a70d-72ec-4c0a-a266-89f7e31d7cbc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.234002 May 25 16:41:23.928660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 939/1856] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:23 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:23.929497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a1258ee4-5b05-4cad-890b-30cafb66ebbf tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.228755 May 25 16:41:23.929965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 918/1857] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:23 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:23.935762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e72d3175-e35f-4f5a-a165-f3fa50b54b46 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:23.936368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e72d3175-e35f-4f5a-a165-f3fa50b54b46 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:23.942705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e72d3175-e35f-4f5a-a165-f3fa50b54b46 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:23.943017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-42e6ef95-df9c-4490-abbc-ec97ebdea6b9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:23.943390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e72d3175-e35f-4f5a-a165-f3fa50b54b46 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:23.943681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-42e6ef95-df9c-4490-abbc-ec97ebdea6b9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:23.944361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e72d3175-e35f-4f5a-a165-f3fa50b54b46 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:23.944920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e72d3175-e35f-4f5a-a165-f3fa50b54b46 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:23.949983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-42e6ef95-df9c-4490-abbc-ec97ebdea6b9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:23.950540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-42e6ef95-df9c-4490-abbc-ec97ebdea6b9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:23.951392 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-42e6ef95-df9c-4490-abbc-ec97ebdea6b9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:23.951824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-42e6ef95-df9c-4490-abbc-ec97ebdea6b9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:24.034838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-42e6ef95-df9c-4490-abbc-ec97ebdea6b9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:24.043700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-42e6ef95-df9c-4490-abbc-ec97ebdea6b9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.114018 May 25 16:41:24.044258 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 940/1858] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:23 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:24.101445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e72d3175-e35f-4f5a-a165-f3fa50b54b46 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:24.109392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e72d3175-e35f-4f5a-a165-f3fa50b54b46 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.178647 May 25 16:41:24.109943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 919/1859] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:23 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:24.950884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-21b6a54f-66c6-4ff6-82a7-dad46197a3b3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:24.951921 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-21b6a54f-66c6-4ff6-82a7-dad46197a3b3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:24.961516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:24.962479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:24.964125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21b6a54f-66c6-4ff6-82a7-dad46197a3b3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:24.964581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21b6a54f-66c6-4ff6-82a7-dad46197a3b3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:24.965361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21b6a54f-66c6-4ff6-82a7-dad46197a3b3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:24.965834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-21b6a54f-66c6-4ff6-82a7-dad46197a3b3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:24.969999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:24.970676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:24.971498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:24.971932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:25.157822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:25.161119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-21b6a54f-66c6-4ff6-82a7-dad46197a3b3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:25.167087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:25.168864 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:25.169982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7df69315-c3e4-4eeb-9a93-48b28e603a89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.214891 May 25 16:41:25.170568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 920/1860] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:24 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:41:25.170727 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-21b6a54f-66c6-4ff6-82a7-dad46197a3b3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.226487 May 25 16:41:25.180844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 941/1861] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:24 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:25.181052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-53f835f1-9acf-46f0-93ae-609641600013 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:25.181162 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-528a7ed2-be4a-4eef-9142-cc6ea922e98d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:25.181259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-53f835f1-9acf-46f0-93ae-609641600013 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:25.181357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-528a7ed2-be4a-4eef-9142-cc6ea922e98d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:25.182320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53f835f1-9acf-46f0-93ae-609641600013 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:25.182765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53f835f1-9acf-46f0-93ae-609641600013 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:25.182926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-528a7ed2-be4a-4eef-9142-cc6ea922e98d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:25.183225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-528a7ed2-be4a-4eef-9142-cc6ea922e98d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:25.183751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53f835f1-9acf-46f0-93ae-609641600013 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:25.183985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-528a7ed2-be4a-4eef-9142-cc6ea922e98d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:25.184177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53f835f1-9acf-46f0-93ae-609641600013 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:25.184419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-528a7ed2-be4a-4eef-9142-cc6ea922e98d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:25.270887 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-53f835f1-9acf-46f0-93ae-609641600013 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:25.279770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-53f835f1-9acf-46f0-93ae-609641600013 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.108232 May 25 16:41:25.280331 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 921/1862] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:25 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:25.345322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-528a7ed2-be4a-4eef-9142-cc6ea922e98d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:25.356576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-528a7ed2-be4a-4eef-9142-cc6ea922e98d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.184049 May 25 16:41:25.357154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 942/1863] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:25 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:26.188300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-15b1c880-46e2-4a71-8419-0ee9b00021b0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:26.188999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:26.189393 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-15b1c880-46e2-4a71-8419-0ee9b00021b0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:26.189505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:26.195748 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15b1c880-46e2-4a71-8419-0ee9b00021b0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:26.195932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:26.196205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15b1c880-46e2-4a71-8419-0ee9b00021b0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:26.196324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:26.196991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15b1c880-46e2-4a71-8419-0ee9b00021b0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:26.197175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:26.197422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15b1c880-46e2-4a71-8419-0ee9b00021b0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:26.197577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:26.396260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-15b1c880-46e2-4a71-8419-0ee9b00021b0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:26.399689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:26.404836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-15b1c880-46e2-4a71-8419-0ee9b00021b0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220430 May 25 16:41:26.405624 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 943/1864] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:26 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:26.410204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-18b4e81a-63b1-4288-84c6-aa27eee8bfe9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:26.410807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-18b4e81a-63b1-4288-84c6-aa27eee8bfe9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:26.410979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:26.412556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:26.413755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fc79823f-c989-4914-a52f-87eb4456767e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.229429 May 25 16:41:26.414938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 922/1865] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:26.417139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-18b4e81a-63b1-4288-84c6-aa27eee8bfe9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:26.417793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-18b4e81a-63b1-4288-84c6-aa27eee8bfe9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:26.418554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-18b4e81a-63b1-4288-84c6-aa27eee8bfe9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:26.418980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-18b4e81a-63b1-4288-84c6-aa27eee8bfe9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:26.419378 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0527bb98-207a-4b04-b1ca-b04d3b0e44c2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:26.419942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0527bb98-207a-4b04-b1ca-b04d3b0e44c2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:26.426045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0527bb98-207a-4b04-b1ca-b04d3b0e44c2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:26.426497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0527bb98-207a-4b04-b1ca-b04d3b0e44c2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:26.427288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0527bb98-207a-4b04-b1ca-b04d3b0e44c2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:26.427722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0527bb98-207a-4b04-b1ca-b04d3b0e44c2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:26.503936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-18b4e81a-63b1-4288-84c6-aa27eee8bfe9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:26.513261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-18b4e81a-63b1-4288-84c6-aa27eee8bfe9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.106841 May 25 16:41:26.513880 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 944/1866] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:26.573448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0527bb98-207a-4b04-b1ca-b04d3b0e44c2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:26.582114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0527bb98-207a-4b04-b1ca-b04d3b0e44c2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.166185 May 25 16:41:26.582752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 923/1867] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:26 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:27.422285 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e6033aa7-2554-4227-b14d-5ec9e9913519 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:27.422900 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e6033aa7-2554-4227-b14d-5ec9e9913519 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:27.429706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6033aa7-2554-4227-b14d-5ec9e9913519 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:27.430158 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6033aa7-2554-4227-b14d-5ec9e9913519 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:27.431075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6033aa7-2554-4227-b14d-5ec9e9913519 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:27.431535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e6033aa7-2554-4227-b14d-5ec9e9913519 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:27.433548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:27.434325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:27.441369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:27.441824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:27.442603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:27.442994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:27.649272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e6033aa7-2554-4227-b14d-5ec9e9913519 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:27.656451 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:27.663903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e6033aa7-2554-4227-b14d-5ec9e9913519 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.245113 May 25 16:41:27.665156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 945/1868] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:27 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:27.670268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0dfd5a9f-84a6-41a8-bdd5-3490325fcc91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:27.670896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0dfd5a9f-84a6-41a8-bdd5-3490325fcc91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:27.676304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:27.676741 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0dfd5a9f-84a6-41a8-bdd5-3490325fcc91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:27.677149 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0dfd5a9f-84a6-41a8-bdd5-3490325fcc91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:27.677894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0dfd5a9f-84a6-41a8-bdd5-3490325fcc91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:27.678021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:27.678317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0dfd5a9f-84a6-41a8-bdd5-3490325fcc91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:27.679314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7ecf7ded-4831-4522-821a-9af27d426be2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.249877 May 25 16:41:27.680081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 924/1869] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:27.684752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-07838845-a6b6-4ef2-a5a4-b07432f6a6fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:27.685354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-07838845-a6b6-4ef2-a5a4-b07432f6a6fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:27.691861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-07838845-a6b6-4ef2-a5a4-b07432f6a6fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:27.693079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-07838845-a6b6-4ef2-a5a4-b07432f6a6fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:27.693079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-07838845-a6b6-4ef2-a5a4-b07432f6a6fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:27.693481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-07838845-a6b6-4ef2-a5a4-b07432f6a6fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:27.745863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0dfd5a9f-84a6-41a8-bdd5-3490325fcc91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:27.754914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0dfd5a9f-84a6-41a8-bdd5-3490325fcc91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088807 May 25 16:41:27.756288 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 946/1870] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:27 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:27.841358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-07838845-a6b6-4ef2-a5a4-b07432f6a6fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:27.854040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-07838845-a6b6-4ef2-a5a4-b07432f6a6fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172575 May 25 16:41:27.854881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 925/1871] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:27 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:28.685180 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a4f05d05-e8a4-44ca-8dc4-2ae1cdbe647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:28.685789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a4f05d05-e8a4-44ca-8dc4-2ae1cdbe647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:28.692916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4f05d05-e8a4-44ca-8dc4-2ae1cdbe647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:28.693371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4f05d05-e8a4-44ca-8dc4-2ae1cdbe647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:28.694139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4f05d05-e8a4-44ca-8dc4-2ae1cdbe647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:28.694574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a4f05d05-e8a4-44ca-8dc4-2ae1cdbe647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:28.697745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:28.698344 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:28.704915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:28.705368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:28.706128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:28.706561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:28.888046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a4f05d05-e8a4-44ca-8dc4-2ae1cdbe647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:28.892173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:28.896365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a4f05d05-e8a4-44ca-8dc4-2ae1cdbe647d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217124 May 25 16:41:28.896934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 947/1872] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:28 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:28.901010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:28.901369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5b40241c-cb3e-4f55-9e5b-94fb14a3d4d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:28.901973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5b40241c-cb3e-4f55-9e5b-94fb14a3d4d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:28.902511 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:28.903710 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e7e0dd0e-aadf-4484-9754-de18ba02d746 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.209895 May 25 16:41:28.913205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 926/1873] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:28.913409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b40241c-cb3e-4f55-9e5b-94fb14a3d4d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:28.913409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b40241c-cb3e-4f55-9e5b-94fb14a3d4d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:28.913604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cc635c9a-4eb6-47af-b602-3a3ee5ac9389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:28.913693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b40241c-cb3e-4f55-9e5b-94fb14a3d4d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:28.913779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cc635c9a-4eb6-47af-b602-3a3ee5ac9389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:28.913870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5b40241c-cb3e-4f55-9e5b-94fb14a3d4d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:28.915532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cc635c9a-4eb6-47af-b602-3a3ee5ac9389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:28.915981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cc635c9a-4eb6-47af-b602-3a3ee5ac9389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:28.916718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cc635c9a-4eb6-47af-b602-3a3ee5ac9389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:28.917226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cc635c9a-4eb6-47af-b602-3a3ee5ac9389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:28.983064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5b40241c-cb3e-4f55-9e5b-94fb14a3d4d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:28.991500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5b40241c-cb3e-4f55-9e5b-94fb14a3d4d9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093551 May 25 16:41:28.992078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 948/1874] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:29.077688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cc635c9a-4eb6-47af-b602-3a3ee5ac9389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:29.088357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cc635c9a-4eb6-47af-b602-3a3ee5ac9389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.182833 May 25 16:41:29.089074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 927/1875] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:28 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:29.918151 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7d318436-5e2a-4ef9-9b0e-13ee5542faec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:29.918841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7d318436-5e2a-4ef9-9b0e-13ee5542faec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:29.923280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:29.923997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:29.926841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d318436-5e2a-4ef9-9b0e-13ee5542faec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:29.927420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d318436-5e2a-4ef9-9b0e-13ee5542faec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:29.928319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d318436-5e2a-4ef9-9b0e-13ee5542faec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:29.928830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d318436-5e2a-4ef9-9b0e-13ee5542faec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:29.931128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:29.931685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:29.932620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:29.933106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:29.950397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:30.129625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7d318436-5e2a-4ef9-9b0e-13ee5542faec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:30.133794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:30.138289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7d318436-5e2a-4ef9-9b0e-13ee5542faec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.226649 May 25 16:41:30.138993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 949/1876] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:29 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:30.142586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:30.143981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9869867c-7053-4701-9dfa-8dac187000f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:30.144129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:30.144584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9869867c-7053-4701-9dfa-8dac187000f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:30.145312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-69a0f42f-ce5a-4fb9-b8ca-e30afe8181ef tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.226582 May 25 16:41:30.145830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 928/1877] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:29 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:30.150365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9869867c-7053-4701-9dfa-8dac187000f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:30.150536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a5d2478a-ee3f-4f5c-b68a-feba36efe1c0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:30.150814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9869867c-7053-4701-9dfa-8dac187000f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:30.151058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a5d2478a-ee3f-4f5c-b68a-feba36efe1c0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:30.151634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9869867c-7053-4701-9dfa-8dac187000f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:30.152069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9869867c-7053-4701-9dfa-8dac187000f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:30.157133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5d2478a-ee3f-4f5c-b68a-feba36efe1c0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:30.157591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5d2478a-ee3f-4f5c-b68a-feba36efe1c0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:30.158358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5d2478a-ee3f-4f5c-b68a-feba36efe1c0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:30.158802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a5d2478a-ee3f-4f5c-b68a-feba36efe1c0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:30.224475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9869867c-7053-4701-9dfa-8dac187000f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:30.232945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9869867c-7053-4701-9dfa-8dac187000f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092478 May 25 16:41:30.233549 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 950/1878] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:30 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:30.310882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a5d2478a-ee3f-4f5c-b68a-feba36efe1c0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:30.319359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a5d2478a-ee3f-4f5c-b68a-feba36efe1c0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172229 May 25 16:41:30.319997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 929/1879] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:30 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:31.156642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-69b653f7-b3c0-4e4f-a7ee-6757a188f996 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:31.157255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-69b653f7-b3c0-4e4f-a7ee-6757a188f996 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:31.163075 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:31.163712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:31.164020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69b653f7-b3c0-4e4f-a7ee-6757a188f996 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:31.164784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69b653f7-b3c0-4e4f-a7ee-6757a188f996 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:31.165537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69b653f7-b3c0-4e4f-a7ee-6757a188f996 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:31.165967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69b653f7-b3c0-4e4f-a7ee-6757a188f996 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:31.170776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:31.171263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:31.172034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:31.172460 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:31.359576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-69b653f7-b3c0-4e4f-a7ee-6757a188f996 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:31.363515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:31.367930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-69b653f7-b3c0-4e4f-a7ee-6757a188f996 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.215162 May 25 16:41:31.368848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 951/1880] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:31 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:31.372702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:31.373260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5d9fa298-f3ee-436c-9394-c470d9707ac1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:31.373853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5d9fa298-f3ee-436c-9394-c470d9707ac1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:31.374326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:31.375569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8b8d7ed8-294f-4011-94f9-6fc0aed1bf27 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.216317 May 25 16:41:31.376562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 930/1881] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:31 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:31.382689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d9fa298-f3ee-436c-9394-c470d9707ac1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:31.382689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d9fa298-f3ee-436c-9394-c470d9707ac1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:31.382689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d9fa298-f3ee-436c-9394-c470d9707ac1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:31.382689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5d9fa298-f3ee-436c-9394-c470d9707ac1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:31.383132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d4a29f90-d677-421b-9749-2907f0734189 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:31.383422 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d4a29f90-d677-421b-9749-2907f0734189 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:31.391613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4a29f90-d677-421b-9749-2907f0734189 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:31.392072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4a29f90-d677-421b-9749-2907f0734189 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:31.392837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4a29f90-d677-421b-9749-2907f0734189 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:31.393271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d4a29f90-d677-421b-9749-2907f0734189 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:31.443796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5d9fa298-f3ee-436c-9394-c470d9707ac1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:31.452242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5d9fa298-f3ee-436c-9394-c470d9707ac1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.082454 May 25 16:41:31.452837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 952/1882] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:31 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:31.552799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d4a29f90-d677-421b-9749-2907f0734189 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:31.562751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d4a29f90-d677-421b-9749-2907f0734189 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.185279 May 25 16:41:31.563359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 931/1883] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:31 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:32.389741 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0cd4c6fe-08b2-4b86-a4a2-2e12909e45b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:32.390351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0cd4c6fe-08b2-4b86-a4a2-2e12909e45b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:32.393886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:32.394634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:32.397247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0cd4c6fe-08b2-4b86-a4a2-2e12909e45b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:32.397671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0cd4c6fe-08b2-4b86-a4a2-2e12909e45b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:32.398471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0cd4c6fe-08b2-4b86-a4a2-2e12909e45b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:32.398900 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0cd4c6fe-08b2-4b86-a4a2-2e12909e45b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:32.401356 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:32.401808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:32.402602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:32.403023 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:32.601566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0cd4c6fe-08b2-4b86-a4a2-2e12909e45b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:32.604541 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:32.610425 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0cd4c6fe-08b2-4b86-a4a2-2e12909e45b2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.224790 May 25 16:41:32.611453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 953/1884] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:32 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:32.613699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:32.615206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:32.615959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-394b1b81-ed9d-4b28-8fd7-bcb51da03f5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:32.616401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9dd23fe4-2c2f-4d81-ac7e-0d06089efa88 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.226463 May 25 16:41:32.616545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-394b1b81-ed9d-4b28-8fd7-bcb51da03f5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:32.617109 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 932/1885] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:32 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:32.621480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-11735f90-eb54-4d5f-907f-86993cfba3b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:32.622045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-11735f90-eb54-4d5f-907f-86993cfba3b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:32.626028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-394b1b81-ed9d-4b28-8fd7-bcb51da03f5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:32.626482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-394b1b81-ed9d-4b28-8fd7-bcb51da03f5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:32.627279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-394b1b81-ed9d-4b28-8fd7-bcb51da03f5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:32.627715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-394b1b81-ed9d-4b28-8fd7-bcb51da03f5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:32.627944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11735f90-eb54-4d5f-907f-86993cfba3b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:32.628392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11735f90-eb54-4d5f-907f-86993cfba3b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:32.629142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11735f90-eb54-4d5f-907f-86993cfba3b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:32.629567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-11735f90-eb54-4d5f-907f-86993cfba3b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:32.698807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-394b1b81-ed9d-4b28-8fd7-bcb51da03f5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:32.707176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-394b1b81-ed9d-4b28-8fd7-bcb51da03f5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.094661 May 25 16:41:32.708109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 954/1886] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:32 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:32.783141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-11735f90-eb54-4d5f-907f-86993cfba3b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:32.791570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-11735f90-eb54-4d5f-907f-86993cfba3b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173663 May 25 16:41:32.792184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 933/1887] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:32 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:33.632260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b25412e1-2cfb-4adf-acfd-cdd9f404bae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:33.633331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b25412e1-2cfb-4adf-acfd-cdd9f404bae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:33.636966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:33.637577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:33.640998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b25412e1-2cfb-4adf-acfd-cdd9f404bae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:33.641457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b25412e1-2cfb-4adf-acfd-cdd9f404bae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:33.642220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b25412e1-2cfb-4adf-acfd-cdd9f404bae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:33.642664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b25412e1-2cfb-4adf-acfd-cdd9f404bae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:33.644127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:33.644600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:33.645297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:33.645447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:33.645777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:33.836456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b25412e1-2cfb-4adf-acfd-cdd9f404bae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:33.840696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:33.845041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b25412e1-2cfb-4adf-acfd-cdd9f404bae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.219456 May 25 16:41:33.845623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 955/1888] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:33 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:33.849684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:33.850063 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-993ec28b-759b-4f7a-a3e7-a90f48b505f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:33.850662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-993ec28b-759b-4f7a-a3e7-a90f48b505f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:33.851197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:33.852621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-46962a1b-7ddd-4d24-a38f-34fda6ee972e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.221455 May 25 16:41:33.856831 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-993ec28b-759b-4f7a-a3e7-a90f48b505f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:33.857150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 934/1889] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:33 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:33.857261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-993ec28b-759b-4f7a-a3e7-a90f48b505f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:33.858033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-993ec28b-759b-4f7a-a3e7-a90f48b505f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:33.858518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-993ec28b-759b-4f7a-a3e7-a90f48b505f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:33.861893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1498746c-00f7-4f42-b629-cff72b3ff234 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:33.862586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1498746c-00f7-4f42-b629-cff72b3ff234 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:33.868644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1498746c-00f7-4f42-b629-cff72b3ff234 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:33.869167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1498746c-00f7-4f42-b629-cff72b3ff234 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:33.870006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1498746c-00f7-4f42-b629-cff72b3ff234 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:33.870576 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1498746c-00f7-4f42-b629-cff72b3ff234 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:33.928634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-993ec28b-759b-4f7a-a3e7-a90f48b505f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:33.937943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-993ec28b-759b-4f7a-a3e7-a90f48b505f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091323 May 25 16:41:33.938514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 956/1890] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:33 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:34.021018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1498746c-00f7-4f42-b629-cff72b3ff234 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:34.029561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1498746c-00f7-4f42-b629-cff72b3ff234 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.171258 May 25 16:41:34.030136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 935/1891] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:33 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:34.866384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-11cff782-9a2a-4d46-a03a-2424411038c4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:34.867457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-11cff782-9a2a-4d46-a03a-2424411038c4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:34.872515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:34.873157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:34.875563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-11cff782-9a2a-4d46-a03a-2424411038c4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:34.876005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-11cff782-9a2a-4d46-a03a-2424411038c4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:34.876760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-11cff782-9a2a-4d46-a03a-2424411038c4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:34.877189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-11cff782-9a2a-4d46-a03a-2424411038c4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:34.879947 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:34.880430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:34.881188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:34.881622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:35.073319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-11cff782-9a2a-4d46-a03a-2424411038c4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:35.076824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:35.081938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-11cff782-9a2a-4d46-a03a-2424411038c4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222053 May 25 16:41:35.082529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 957/1892] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:34 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:35.086217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:35.087028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c7a79ada-9d8e-48a2-b7fb-f8916466540a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:35.087752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:35.087917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c7a79ada-9d8e-48a2-b7fb-f8916466540a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:35.088908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-717f2abb-9a2a-474f-81ae-c1e0b7197016 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.221594 May 25 16:41:35.091130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 936/1893] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:34 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:35.094045 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2ef103aa-088b-4ee7-8543-83d3ee5a737b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:35.094594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7a79ada-9d8e-48a2-b7fb-f8916466540a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:35.094710 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2ef103aa-088b-4ee7-8543-83d3ee5a737b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:35.095046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7a79ada-9d8e-48a2-b7fb-f8916466540a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:35.095829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7a79ada-9d8e-48a2-b7fb-f8916466540a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:35.096264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7a79ada-9d8e-48a2-b7fb-f8916466540a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:35.100587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ef103aa-088b-4ee7-8543-83d3ee5a737b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:35.101038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ef103aa-088b-4ee7-8543-83d3ee5a737b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:35.101941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ef103aa-088b-4ee7-8543-83d3ee5a737b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:35.102375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2ef103aa-088b-4ee7-8543-83d3ee5a737b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:35.163895 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c7a79ada-9d8e-48a2-b7fb-f8916466540a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:35.172929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c7a79ada-9d8e-48a2-b7fb-f8916466540a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089432 May 25 16:41:35.176172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 958/1894] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:35 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:35.245280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2ef103aa-088b-4ee7-8543-83d3ee5a737b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:35.253330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2ef103aa-088b-4ee7-8543-83d3ee5a737b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162779 May 25 16:41:35.253868 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 937/1895] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:35 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:36.102120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c5444cc8-52af-4f8d-820c-090e6d17fd46 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:36.102721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c5444cc8-52af-4f8d-820c-090e6d17fd46 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:36.108788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:36.109468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:36.109781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5444cc8-52af-4f8d-820c-090e6d17fd46 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:36.110236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5444cc8-52af-4f8d-820c-090e6d17fd46 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:36.111007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5444cc8-52af-4f8d-820c-090e6d17fd46 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:36.111476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5444cc8-52af-4f8d-820c-090e6d17fd46 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:36.115938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:36.116404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:36.117127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:36.117558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:36.303569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:36.306382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c5444cc8-52af-4f8d-820c-090e6d17fd46 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:36.312428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:36.313933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:36.314961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c5444cc8-52af-4f8d-820c-090e6d17fd46 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.219075 May 25 16:41:36.315100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-067e150e-8eb0-48ab-aa68-4f592b974e93 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.210284 May 25 16:41:36.315828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 959/1896] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:36 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:36.316116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 938/1897] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:36 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:36.320605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b9fd9acc-7ef9-495b-9d2d-5a7cc018fb89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:36.321203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b9fd9acc-7ef9-495b-9d2d-5a7cc018fb89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:36.327482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9fd9acc-7ef9-495b-9d2d-5a7cc018fb89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:36.327876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9fd9acc-7ef9-495b-9d2d-5a7cc018fb89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:36.328632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9fd9acc-7ef9-495b-9d2d-5a7cc018fb89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:36.329061 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9fd9acc-7ef9-495b-9d2d-5a7cc018fb89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:36.331424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bc97f880-6287-40dc-ad44-8b48a9b16e20 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:36.332118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bc97f880-6287-40dc-ad44-8b48a9b16e20 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:36.339449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc97f880-6287-40dc-ad44-8b48a9b16e20 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:36.339902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc97f880-6287-40dc-ad44-8b48a9b16e20 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:36.340676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc97f880-6287-40dc-ad44-8b48a9b16e20 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:36.341113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc97f880-6287-40dc-ad44-8b48a9b16e20 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:36.426607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bc97f880-6287-40dc-ad44-8b48a9b16e20 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:36.435123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bc97f880-6287-40dc-ad44-8b48a9b16e20 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.118493 May 25 16:41:36.435745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 960/1898] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:36 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:36.493237 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b9fd9acc-7ef9-495b-9d2d-5a7cc018fb89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:36.506283 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b9fd9acc-7ef9-495b-9d2d-5a7cc018fb89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.188923 May 25 16:41:36.507048 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 939/1899] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:36 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:37.342672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:37.343586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:37.348899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-621093f5-86ec-41b1-805d-f55abd740911 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:37.349908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-621093f5-86ec-41b1-805d-f55abd740911 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:37.354617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:37.355065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:37.355862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:37.356298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:37.358165 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-621093f5-86ec-41b1-805d-f55abd740911 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:37.358620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-621093f5-86ec-41b1-805d-f55abd740911 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:37.359396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-621093f5-86ec-41b1-805d-f55abd740911 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:37.359836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-621093f5-86ec-41b1-805d-f55abd740911 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:37.558281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:37.562058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-621093f5-86ec-41b1-805d-f55abd740911 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:37.567089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:37.568677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:37.569858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-acec9989-018b-4858-9294-34b8e4fe4165 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.234426 May 25 16:41:37.570145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-621093f5-86ec-41b1-805d-f55abd740911 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.228206 May 25 16:41:37.570457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 961/1900] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:37 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:37.577032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 940/1901] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:37 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:37.577218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e0917ee5-c9da-49ca-b37e-4d1778d11faa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:37.577218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e0917ee5-c9da-49ca-b37e-4d1778d11faa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:37.577418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3c287eb1-ef04-4177-93cd-102f5d26fafe tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:37.577418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3c287eb1-ef04-4177-93cd-102f5d26fafe tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:37.582041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c287eb1-ef04-4177-93cd-102f5d26fafe tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:37.582536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c287eb1-ef04-4177-93cd-102f5d26fafe tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:37.583284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c287eb1-ef04-4177-93cd-102f5d26fafe tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:37.583709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3c287eb1-ef04-4177-93cd-102f5d26fafe tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:37.584866 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0917ee5-c9da-49ca-b37e-4d1778d11faa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:37.585321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0917ee5-c9da-49ca-b37e-4d1778d11faa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:37.586083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0917ee5-c9da-49ca-b37e-4d1778d11faa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:37.586521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e0917ee5-c9da-49ca-b37e-4d1778d11faa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:37.669954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e0917ee5-c9da-49ca-b37e-4d1778d11faa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:37.678519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e0917ee5-c9da-49ca-b37e-4d1778d11faa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.107099 May 25 16:41:37.679092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 962/1902] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:37 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:37.736707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3c287eb1-ef04-4177-93cd-102f5d26fafe tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:37.744994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3c287eb1-ef04-4177-93cd-102f5d26fafe tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172838 May 25 16:41:37.745558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 941/1903] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:37 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:38.587098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6fe61f64-2246-48bb-a4fc-bbb4984f0c5a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:38.587708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6fe61f64-2246-48bb-a4fc-bbb4984f0c5a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:38.588007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:38.588689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:38.594820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fe61f64-2246-48bb-a4fc-bbb4984f0c5a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:38.595297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fe61f64-2246-48bb-a4fc-bbb4984f0c5a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:38.595702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:38.596055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fe61f64-2246-48bb-a4fc-bbb4984f0c5a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:38.596208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:38.596486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fe61f64-2246-48bb-a4fc-bbb4984f0c5a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:38.597058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:38.597491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:38.785457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6fe61f64-2246-48bb-a4fc-bbb4984f0c5a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:38.788741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:38.794004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6fe61f64-2246-48bb-a4fc-bbb4984f0c5a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.210972 May 25 16:41:38.794792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 963/1904] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:38 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:38.798330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:38.799090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b1b9a85a-0bf2-4c73-a607-f9325d21c32a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:38.799717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b1b9a85a-0bf2-4c73-a607-f9325d21c32a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:38.799872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:38.801154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ccd3deb6-6771-4768-9608-8744993f02a8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.216913 May 25 16:41:38.803616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 942/1905] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:38 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:38.806278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-469cd980-a4ac-40d6-b119-7ef92c3debe4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:38.806869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-469cd980-a4ac-40d6-b119-7ef92c3debe4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:38.806977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b1b9a85a-0bf2-4c73-a607-f9325d21c32a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:38.807438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b1b9a85a-0bf2-4c73-a607-f9325d21c32a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:38.808452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b1b9a85a-0bf2-4c73-a607-f9325d21c32a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:38.808876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b1b9a85a-0bf2-4c73-a607-f9325d21c32a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:38.813213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-469cd980-a4ac-40d6-b119-7ef92c3debe4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:38.813663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-469cd980-a4ac-40d6-b119-7ef92c3debe4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:38.814426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-469cd980-a4ac-40d6-b119-7ef92c3debe4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:38.814860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-469cd980-a4ac-40d6-b119-7ef92c3debe4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:38.895529 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b1b9a85a-0bf2-4c73-a607-f9325d21c32a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:38.908662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b1b9a85a-0bf2-4c73-a607-f9325d21c32a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.112947 May 25 16:41:38.909321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 964/1906] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:38 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:38.966510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-469cd980-a4ac-40d6-b119-7ef92c3debe4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:38.978545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-469cd980-a4ac-40d6-b119-7ef92c3debe4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.175483 May 25 16:41:38.979737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 943/1907] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:38 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:39.815662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d6a7804c-5036-47a7-aa48-c6c5f6ca9363 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:39.816608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d6a7804c-5036-47a7-aa48-c6c5f6ca9363 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:39.822956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:39.824125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:39.827775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6a7804c-5036-47a7-aa48-c6c5f6ca9363 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:39.828559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6a7804c-5036-47a7-aa48-c6c5f6ca9363 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:39.829824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6a7804c-5036-47a7-aa48-c6c5f6ca9363 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:39.830556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6a7804c-5036-47a7-aa48-c6c5f6ca9363 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:39.834612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:39.835479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:39.836713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:39.837413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:40.267108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d6a7804c-5036-47a7-aa48-c6c5f6ca9363 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:40.270341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:40.275668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d6a7804c-5036-47a7-aa48-c6c5f6ca9363 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.466697 May 25 16:41:40.276414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 965/1908] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:39 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:41:40.279310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:40.280810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:40.280945 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-94f8e955-7cd4-48d9-9e89-7d026b563ab3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:40.281531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-94f8e955-7cd4-48d9-9e89-7d026b563ab3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:40.281971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-90fa997a-ae9e-48c6-8a0b-3a88a7ac922a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.465465 May 25 16:41:40.282705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 944/1909] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:39 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:40.286899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7743e46d-fffb-45b9-b69b-b57fbc7ea867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:40.287535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7743e46d-fffb-45b9-b69b-b57fbc7ea867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:40.287670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94f8e955-7cd4-48d9-9e89-7d026b563ab3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:40.288197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94f8e955-7cd4-48d9-9e89-7d026b563ab3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:40.289036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94f8e955-7cd4-48d9-9e89-7d026b563ab3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:40.289852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-94f8e955-7cd4-48d9-9e89-7d026b563ab3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:40.294026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7743e46d-fffb-45b9-b69b-b57fbc7ea867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:40.294443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7743e46d-fffb-45b9-b69b-b57fbc7ea867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:40.295191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7743e46d-fffb-45b9-b69b-b57fbc7ea867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:40.295645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7743e46d-fffb-45b9-b69b-b57fbc7ea867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:40.298326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:40.372893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-94f8e955-7cd4-48d9-9e89-7d026b563ab3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:40.381317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-94f8e955-7cd4-48d9-9e89-7d026b563ab3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.103939 May 25 16:41:40.381904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 966/1910] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:40 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:40.444448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7743e46d-fffb-45b9-b69b-b57fbc7ea867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:40.453898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7743e46d-fffb-45b9-b69b-b57fbc7ea867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.170431 May 25 16:41:40.454434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 945/1911] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:40 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:41.304619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:41.305634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:41.308396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a08f04d2-ac98-48b5-9cc8-ee0512cc641c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:41.309008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a08f04d2-ac98-48b5-9cc8-ee0512cc641c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:41.311319 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=96 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:41.313601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:41.314046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:41.314821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:41.315286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:41.316940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a08f04d2-ac98-48b5-9cc8-ee0512cc641c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:41.317391 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a08f04d2-ac98-48b5-9cc8-ee0512cc641c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:41.318151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a08f04d2-ac98-48b5-9cc8-ee0512cc641c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:41.318584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a08f04d2-ac98-48b5-9cc8-ee0512cc641c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:41.503169 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:41.510365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a08f04d2-ac98-48b5-9cc8-ee0512cc641c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:41.512147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:41.513848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:41.515033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b0191021-df1b-49f9-8f97-6bcde4d43a12 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.216897 May 25 16:41:41.515624 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 967/1912] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:41.518810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a08f04d2-ac98-48b5-9cc8-ee0512cc641c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217498 May 25 16:41:41.519583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 946/1913] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:41 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:41.519994 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2a010171-6b4a-4676-adf5-e117b528777c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:41.520595 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2a010171-6b4a-4676-adf5-e117b528777c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:41.526576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a010171-6b4a-4676-adf5-e117b528777c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:41.527197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a010171-6b4a-4676-adf5-e117b528777c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:41.527943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a010171-6b4a-4676-adf5-e117b528777c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:41.528385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a010171-6b4a-4676-adf5-e117b528777c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:41.530612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2466f24d-f9c1-40a2-8589-5c9549568d19 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:41.531245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2466f24d-f9c1-40a2-8589-5c9549568d19 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:41.537250 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2466f24d-f9c1-40a2-8589-5c9549568d19 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:41.537699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2466f24d-f9c1-40a2-8589-5c9549568d19 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:41.538455 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2466f24d-f9c1-40a2-8589-5c9549568d19 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:41.538856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2466f24d-f9c1-40a2-8589-5c9549568d19 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:41.595851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2a010171-6b4a-4676-adf5-e117b528777c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:41.604813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2a010171-6b4a-4676-adf5-e117b528777c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088072 May 25 16:41:41.605753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 968/1914] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:41 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:41.837838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2466f24d-f9c1-40a2-8589-5c9549568d19 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:41.845948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2466f24d-f9c1-40a2-8589-5c9549568d19 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.318929 May 25 16:41:41.846530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 947/1915] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:41 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:42.536614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:42.537548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:42.539599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6a0a1946-b4ee-483c-8ed4-34215beaca7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:42.540540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6a0a1946-b4ee-483c-8ed4-34215beaca7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:42.546105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:42.546564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:42.547365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:42.547806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:42.549625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a0a1946-b4ee-483c-8ed4-34215beaca7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:42.550384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a0a1946-b4ee-483c-8ed4-34215beaca7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:42.551280 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a0a1946-b4ee-483c-8ed4-34215beaca7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:42.553013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6a0a1946-b4ee-483c-8ed4-34215beaca7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:42.739045 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:42.743308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6a0a1946-b4ee-483c-8ed4-34215beaca7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:42.748251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:42.749763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:42.750941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d0c9f51a-a83c-4ac0-a241-f99d2b94187a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.220603 May 25 16:41:42.751568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 969/1916] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:42.753336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6a0a1946-b4ee-483c-8ed4-34215beaca7f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.218405 May 25 16:41:42.753336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 948/1917] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:42 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:42.756670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c4d9abc9-d035-45f7-a3c2-e53764e52243 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:42.757527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c4d9abc9-d035-45f7-a3c2-e53764e52243 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:42.763997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4d9abc9-d035-45f7-a3c2-e53764e52243 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:42.764453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4d9abc9-d035-45f7-a3c2-e53764e52243 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:42.765214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4d9abc9-d035-45f7-a3c2-e53764e52243 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:42.765662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4d9abc9-d035-45f7-a3c2-e53764e52243 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:42.830398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c4d9abc9-d035-45f7-a3c2-e53764e52243 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:42.838578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c4d9abc9-d035-45f7-a3c2-e53764e52243 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085911 May 25 16:41:42.839362 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 970/1918] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:42 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:42.863062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8baf1ace-0226-4dd3-90f9-8e70d677decc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:42.863701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8baf1ace-0226-4dd3-90f9-8e70d677decc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:42.869759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8baf1ace-0226-4dd3-90f9-8e70d677decc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:42.870207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8baf1ace-0226-4dd3-90f9-8e70d677decc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:42.870975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8baf1ace-0226-4dd3-90f9-8e70d677decc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:42.871430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8baf1ace-0226-4dd3-90f9-8e70d677decc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:43.006083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8baf1ace-0226-4dd3-90f9-8e70d677decc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:43.014653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8baf1ace-0226-4dd3-90f9-8e70d677decc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.155412 May 25 16:41:43.015211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 949/1919] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:42 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:43.773179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a9c306f1-1510-4f9d-a7c8-4ae82024f407 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:43.774109 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:43.774710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a9c306f1-1510-4f9d-a7c8-4ae82024f407 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:43.774920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:43.781908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-1510-4f9d-a7c8-4ae82024f407 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:43.782104 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:43.782363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-1510-4f9d-a7c8-4ae82024f407 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:43.782506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:43.783126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:43.783296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-1510-4f9d-a7c8-4ae82024f407 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:43.783596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-1510-4f9d-a7c8-4ae82024f407 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:43.783761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:43.974534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a9c306f1-1510-4f9d-a7c8-4ae82024f407 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:43.976790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:43.977985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:43.984138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a9c306f1-1510-4f9d-a7c8-4ae82024f407 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217345 May 25 16:41:43.984690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 971/1920] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:43 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:43.986715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:43.988257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:43.989071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ac10d151-359e-4e20-b0cb-5dd3780078d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:43.989423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-757ce1d8-b322-490f-a1ab-3371e5dfee89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.222724 May 25 16:41:43.989674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ac10d151-359e-4e20-b0cb-5dd3780078d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:43.990164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 950/1921] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:43.996062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac10d151-359e-4e20-b0cb-5dd3780078d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:43.996515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac10d151-359e-4e20-b0cb-5dd3780078d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:43.997279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac10d151-359e-4e20-b0cb-5dd3780078d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:43.997710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac10d151-359e-4e20-b0cb-5dd3780078d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:44.032708 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ae7e4a02-362e-4837-9403-2cd7005bb527 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:44.033198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ae7e4a02-362e-4837-9403-2cd7005bb527 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:44.040321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae7e4a02-362e-4837-9403-2cd7005bb527 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:44.040777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae7e4a02-362e-4837-9403-2cd7005bb527 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:44.041602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae7e4a02-362e-4837-9403-2cd7005bb527 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:44.042029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ae7e4a02-362e-4837-9403-2cd7005bb527 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:44.065323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ac10d151-359e-4e20-b0cb-5dd3780078d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:44.073898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ac10d151-359e-4e20-b0cb-5dd3780078d6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088193 May 25 16:41:44.074555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 972/1922] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:43 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:44.176792 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ae7e4a02-362e-4837-9403-2cd7005bb527 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:44.185146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ae7e4a02-362e-4837-9403-2cd7005bb527 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.156378 May 25 16:41:44.185717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 951/1923] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:44 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:45.005820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-14f00601-212d-4ab3-82e8-be5bcdde28e9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:45.006805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-14f00601-212d-4ab3-82e8-be5bcdde28e9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:45.011137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:45.011794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:45.015086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-14f00601-212d-4ab3-82e8-be5bcdde28e9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:45.015590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-14f00601-212d-4ab3-82e8-be5bcdde28e9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:45.016360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-14f00601-212d-4ab3-82e8-be5bcdde28e9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:45.016803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-14f00601-212d-4ab3-82e8-be5bcdde28e9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:45.018491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:45.018937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:45.019743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:45.020174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:45.208902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-14f00601-212d-4ab3-82e8-be5bcdde28e9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:45.212993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:45.217207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-14f00601-212d-4ab3-82e8-be5bcdde28e9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.218117 May 25 16:41:45.217790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 973/1924] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:44 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:45.221834 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:45.222417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0bd9cf84-48cd-4e27-9804-923e42967843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:45.223043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0bd9cf84-48cd-4e27-9804-923e42967843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:45.223393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:45.224646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b4e1ed5d-0303-4fe9-b24a-30f595782bfd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.219676 May 25 16:41:45.225351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 952/1925] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:45 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:45.229744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-747993e7-0687-4bb8-abf2-430d2b4acb1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:45.230382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-747993e7-0687-4bb8-abf2-430d2b4acb1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:45.236039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bd9cf84-48cd-4e27-9804-923e42967843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:45.236286 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-747993e7-0687-4bb8-abf2-430d2b4acb1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:45.236497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bd9cf84-48cd-4e27-9804-923e42967843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:45.236761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-747993e7-0687-4bb8-abf2-430d2b4acb1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:45.237265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bd9cf84-48cd-4e27-9804-923e42967843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:45.237494 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-747993e7-0687-4bb8-abf2-430d2b4acb1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:45.237694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0bd9cf84-48cd-4e27-9804-923e42967843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:45.237930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-747993e7-0687-4bb8-abf2-430d2b4acb1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:45.324660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0bd9cf84-48cd-4e27-9804-923e42967843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:45.333331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0bd9cf84-48cd-4e27-9804-923e42967843 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.114448 May 25 16:41:45.333925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 974/1926] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:45 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:45.398329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-747993e7-0687-4bb8-abf2-430d2b4acb1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:45.407143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-747993e7-0687-4bb8-abf2-430d2b4acb1c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.180918 May 25 16:41:45.411385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 953/1927] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:45 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:46.238695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-478b1702-d901-4273-8981-38d5fc72b483 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:46.239744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-478b1702-d901-4273-8981-38d5fc72b483 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:46.247078 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:46.248137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:46.250729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-478b1702-d901-4273-8981-38d5fc72b483 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:46.251569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-478b1702-d901-4273-8981-38d5fc72b483 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:46.252839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-478b1702-d901-4273-8981-38d5fc72b483 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:46.253561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-478b1702-d901-4273-8981-38d5fc72b483 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:46.258870 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:46.259645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:46.260731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:46.261164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:46.457277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-478b1702-d901-4273-8981-38d5fc72b483 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:46.460246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:46.465584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-478b1702-d901-4273-8981-38d5fc72b483 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.233346 May 25 16:41:46.466210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 975/1928] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:46 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:46.469432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:46.470626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e5f98dd0-9659-4749-8529-8ac25abde702 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:46.471274 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e5f98dd0-9659-4749-8529-8ac25abde702 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:46.471435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:46.472526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e795aadf-8793-407a-ade3-eb796ed9c805 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.231952 May 25 16:41:46.473606 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 954/1929] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:46 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:46.477302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5f98dd0-9659-4749-8529-8ac25abde702 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:46.477756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5f98dd0-9659-4749-8529-8ac25abde702 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:46.478236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e954843d-d4cc-4c0c-b029-83bb5ad623d1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:46.478511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5f98dd0-9659-4749-8529-8ac25abde702 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:46.478948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e954843d-d4cc-4c0c-b029-83bb5ad623d1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:46.479066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5f98dd0-9659-4749-8529-8ac25abde702 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:46.485755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e954843d-d4cc-4c0c-b029-83bb5ad623d1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:46.486196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e954843d-d4cc-4c0c-b029-83bb5ad623d1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:46.486958 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e954843d-d4cc-4c0c-b029-83bb5ad623d1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:46.487404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e954843d-d4cc-4c0c-b029-83bb5ad623d1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:46.566200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e5f98dd0-9659-4749-8529-8ac25abde702 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:46.574305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e5f98dd0-9659-4749-8529-8ac25abde702 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.107141 May 25 16:41:46.574983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 976/1930] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:46 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:46.637892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e954843d-d4cc-4c0c-b029-83bb5ad623d1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:46.650306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e954843d-d4cc-4c0c-b029-83bb5ad623d1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.175111 May 25 16:41:46.651084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 955/1931] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:46 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:47.487385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-64c46d65-3b81-4eee-b7f0-9de56534f087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:47.488360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-64c46d65-3b81-4eee-b7f0-9de56534f087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:47.492137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=96 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:47.495370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:47.496077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:47.498718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64c46d65-3b81-4eee-b7f0-9de56534f087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:47.499123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64c46d65-3b81-4eee-b7f0-9de56534f087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:47.499946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64c46d65-3b81-4eee-b7f0-9de56534f087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:47.500372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64c46d65-3b81-4eee-b7f0-9de56534f087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:47.502987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:47.503472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:47.504240 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:47.504691 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:47.693146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-64c46d65-3b81-4eee-b7f0-9de56534f087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:47.696203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:47.701541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-64c46d65-3b81-4eee-b7f0-9de56534f087 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220733 May 25 16:41:47.702154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 977/1932] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:47 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:47.704957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:47.706453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:47.706621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-25c9381e-0661-4d6e-8c29-fbc2e77660df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:47.707214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-25c9381e-0661-4d6e-8c29-fbc2e77660df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:47.707663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5cb038d3-2a55-44b0-b9ee-415533aa01f0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.218832 May 25 16:41:47.708332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 956/1933] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:47.718184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-25c9381e-0661-4d6e-8c29-fbc2e77660df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:47.718184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-25c9381e-0661-4d6e-8c29-fbc2e77660df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:47.718184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-25c9381e-0661-4d6e-8c29-fbc2e77660df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:47.718184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-25c9381e-0661-4d6e-8c29-fbc2e77660df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:47.718589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aa1c3eeb-b2f8-4c09-a4a2-67c7b58da7e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:47.718589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aa1c3eeb-b2f8-4c09-a4a2-67c7b58da7e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:47.724880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa1c3eeb-b2f8-4c09-a4a2-67c7b58da7e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:47.725337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa1c3eeb-b2f8-4c09-a4a2-67c7b58da7e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:47.726114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa1c3eeb-b2f8-4c09-a4a2-67c7b58da7e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:47.726547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa1c3eeb-b2f8-4c09-a4a2-67c7b58da7e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:47.787951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-25c9381e-0661-4d6e-8c29-fbc2e77660df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:47.796322 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-25c9381e-0661-4d6e-8c29-fbc2e77660df tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093253 May 25 16:41:47.796887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 978/1934] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:47 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:47.868949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa1c3eeb-b2f8-4c09-a4a2-67c7b58da7e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:47.877585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aa1c3eeb-b2f8-4c09-a4a2-67c7b58da7e0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.163302 May 25 16:41:47.878162 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 957/1935] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:47 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:48.723005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c4333d81-134e-4fb5-8c8d-fadb6ea175ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:48.723930 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c4333d81-134e-4fb5-8c8d-fadb6ea175ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:48.729547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:48.730163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:48.732993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4333d81-134e-4fb5-8c8d-fadb6ea175ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:48.733430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4333d81-134e-4fb5-8c8d-fadb6ea175ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:48.734178 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4333d81-134e-4fb5-8c8d-fadb6ea175ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:48.734623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c4333d81-134e-4fb5-8c8d-fadb6ea175ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:48.736515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:48.737008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:48.737755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:48.738194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:48.929369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c4333d81-134e-4fb5-8c8d-fadb6ea175ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:48.933071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:48.937847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c4333d81-134e-4fb5-8c8d-fadb6ea175ef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221367 May 25 16:41:48.938462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 979/1936] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:48 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:48.942399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:48.942932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0679672f-5c07-4ab5-b700-7340d5d3ea35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:48.943562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0679672f-5c07-4ab5-b700-7340d5d3ea35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:48.944004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:48.945296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fecb2e20-387b-47af-822e-d8557c4d0120 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.222134 May 25 16:41:48.946450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 958/1937] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:48.949801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0679672f-5c07-4ab5-b700-7340d5d3ea35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:48.950508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ccf4b5b8-fb47-4301-95bf-29eb4afb4238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:48.950677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0679672f-5c07-4ab5-b700-7340d5d3ea35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:48.951112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ccf4b5b8-fb47-4301-95bf-29eb4afb4238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:48.951558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0679672f-5c07-4ab5-b700-7340d5d3ea35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:48.952072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0679672f-5c07-4ab5-b700-7340d5d3ea35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:48.957849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccf4b5b8-fb47-4301-95bf-29eb4afb4238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:48.958300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccf4b5b8-fb47-4301-95bf-29eb4afb4238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:48.959063 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccf4b5b8-fb47-4301-95bf-29eb4afb4238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:48.959542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccf4b5b8-fb47-4301-95bf-29eb4afb4238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:49.031406 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0679672f-5c07-4ab5-b700-7340d5d3ea35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:49.040063 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0679672f-5c07-4ab5-b700-7340d5d3ea35 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.100652 May 25 16:41:49.040638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 980/1938] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:48 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:49.120490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ccf4b5b8-fb47-4301-95bf-29eb4afb4238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:49.128906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ccf4b5b8-fb47-4301-95bf-29eb4afb4238 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.182009 May 25 16:41:49.129483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 959/1939] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:48 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:49.959323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-01856956-34c5-4133-94bb-ebd0869f116d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:49.960265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-01856956-34c5-4133-94bb-ebd0869f116d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:49.967941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:49.968889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:49.971479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01856956-34c5-4133-94bb-ebd0869f116d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:49.972259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01856956-34c5-4133-94bb-ebd0869f116d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:49.973434 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01856956-34c5-4133-94bb-ebd0869f116d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:49.974196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-01856956-34c5-4133-94bb-ebd0869f116d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:49.979634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:49.980079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:49.980843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:49.981267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:50.173524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-01856956-34c5-4133-94bb-ebd0869f116d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:50.176772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:50.181790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-01856956-34c5-4133-94bb-ebd0869f116d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.229165 May 25 16:41:50.182369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 981/1940] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:49 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:50.185811 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:50.186926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-51117ac4-d0e3-4d74-b0c9-d8a4f6aaaafe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:50.187687 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-51117ac4-d0e3-4d74-b0c9-d8a4f6aaaafe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:50.187807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:50.189142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-966d0f20-d839-4931-844e-827df7f1d1e8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.228152 May 25 16:41:50.190492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 960/1941] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:41:50.194952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5f49b72e-8081-4e11-a812-b125b3575dae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:50.195634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5f49b72e-8081-4e11-a812-b125b3575dae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:50.197569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-51117ac4-d0e3-4d74-b0c9-d8a4f6aaaafe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:50.198127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-51117ac4-d0e3-4d74-b0c9-d8a4f6aaaafe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:50.198920 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-51117ac4-d0e3-4d74-b0c9-d8a4f6aaaafe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:50.199512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-51117ac4-d0e3-4d74-b0c9-d8a4f6aaaafe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:50.201985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f49b72e-8081-4e11-a812-b125b3575dae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:50.202466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f49b72e-8081-4e11-a812-b125b3575dae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:50.203277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f49b72e-8081-4e11-a812-b125b3575dae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:50.203707 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f49b72e-8081-4e11-a812-b125b3575dae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:50.281680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-51117ac4-d0e3-4d74-b0c9-d8a4f6aaaafe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:50.293667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-51117ac4-d0e3-4d74-b0c9-d8a4f6aaaafe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.109592 May 25 16:41:50.294607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 982/1942] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:50 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:50.364517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5f49b72e-8081-4e11-a812-b125b3575dae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:50.366773 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:50.373792 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5f49b72e-8081-4e11-a812-b125b3575dae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.182285 May 25 16:41:50.374365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 961/1943] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:50 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:51.205107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7a3cf2cd-32aa-4f5f-aea0-2ffb4316a26c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:51.206077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7a3cf2cd-32aa-4f5f-aea0-2ffb4316a26c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:51.211724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:51.212762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:51.215297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a3cf2cd-32aa-4f5f-aea0-2ffb4316a26c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:51.215752 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a3cf2cd-32aa-4f5f-aea0-2ffb4316a26c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:51.216512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a3cf2cd-32aa-4f5f-aea0-2ffb4316a26c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:51.216955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a3cf2cd-32aa-4f5f-aea0-2ffb4316a26c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:51.220323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:51.220795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:51.221545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:51.221980 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:51.408670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7a3cf2cd-32aa-4f5f-aea0-2ffb4316a26c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:51.413018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:51.419615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7a3cf2cd-32aa-4f5f-aea0-2ffb4316a26c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.222307 May 25 16:41:51.420179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 983/1944] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:51 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:51.423065 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:51.424614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:51.425781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-be61bc48-bba7-4e73-aba0-20c8a0f83cf5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.220448 May 25 16:41:51.425974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4f15709a-f5d2-406d-851a-630e446f7042 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:51.426457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 962/1945] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:51.426776 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4f15709a-f5d2-406d-851a-630e446f7042 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:51.432899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4f15709a-f5d2-406d-851a-630e446f7042 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:51.433475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4f15709a-f5d2-406d-851a-630e446f7042 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:51.433715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b5e89206-9dd9-4809-b967-54b08489f12d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:51.434322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b5e89206-9dd9-4809-b967-54b08489f12d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:51.434517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4f15709a-f5d2-406d-851a-630e446f7042 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:51.435049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4f15709a-f5d2-406d-851a-630e446f7042 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:51.440384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5e89206-9dd9-4809-b967-54b08489f12d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:51.440840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5e89206-9dd9-4809-b967-54b08489f12d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:51.441601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5e89206-9dd9-4809-b967-54b08489f12d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:51.442072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5e89206-9dd9-4809-b967-54b08489f12d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:51.519338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4f15709a-f5d2-406d-851a-630e446f7042 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:51.527882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4f15709a-f5d2-406d-851a-630e446f7042 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.105581 May 25 16:41:51.528764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 984/1946] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:51 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:51.588528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b5e89206-9dd9-4809-b967-54b08489f12d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:51.597126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b5e89206-9dd9-4809-b967-54b08489f12d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.166932 May 25 16:41:51.597698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 963/1947] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:51 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:52.443289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-371c7162-b4a1-4afa-82da-8e983a2f24aa tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:52.444457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-371c7162-b4a1-4afa-82da-8e983a2f24aa tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:52.448527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:52.449135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:52.452768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-371c7162-b4a1-4afa-82da-8e983a2f24aa tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:52.453237 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-371c7162-b4a1-4afa-82da-8e983a2f24aa tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:52.454001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-371c7162-b4a1-4afa-82da-8e983a2f24aa tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:52.454453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-371c7162-b4a1-4afa-82da-8e983a2f24aa tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:52.455753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:52.456199 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:52.456951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:52.457364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:52.646208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-371c7162-b4a1-4afa-82da-8e983a2f24aa tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:52.649027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:52.655051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-371c7162-b4a1-4afa-82da-8e983a2f24aa tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220551 May 25 16:41:52.655678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 985/1948] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:52 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:52.660177 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3d771a78-7e2f-431a-9e72-677b57ebc01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:52.660753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3d771a78-7e2f-431a-9e72-677b57ebc01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:52.661315 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:52.663723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:52.664789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-117e8dd4-5ee1-4c38-aa67-5f4880ea508e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.223161 May 25 16:41:52.665349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 964/1949] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:52.667447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3d771a78-7e2f-431a-9e72-677b57ebc01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:52.667942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3d771a78-7e2f-431a-9e72-677b57ebc01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:52.668717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3d771a78-7e2f-431a-9e72-677b57ebc01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:52.669539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3d771a78-7e2f-431a-9e72-677b57ebc01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:52.669949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4c2af794-f46a-4ddd-8022-c12bc1b110ef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:52.670555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4c2af794-f46a-4ddd-8022-c12bc1b110ef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:52.676643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c2af794-f46a-4ddd-8022-c12bc1b110ef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:52.677111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c2af794-f46a-4ddd-8022-c12bc1b110ef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:52.678002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c2af794-f46a-4ddd-8022-c12bc1b110ef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:52.678440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c2af794-f46a-4ddd-8022-c12bc1b110ef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:52.741422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3d771a78-7e2f-431a-9e72-677b57ebc01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:52.749644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3d771a78-7e2f-431a-9e72-677b57ebc01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093044 May 25 16:41:52.750213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 986/1950] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:52 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:52.836765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4c2af794-f46a-4ddd-8022-c12bc1b110ef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:52.845377 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4c2af794-f46a-4ddd-8022-c12bc1b110ef tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.179102 May 25 16:41:52.845982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 965/1951] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:52 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:53.677039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-856b8772-ef57-4b24-bae7-d64f3051beb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:53.678104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-856b8772-ef57-4b24-bae7-d64f3051beb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:53.687358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:53.688359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:53.690844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-856b8772-ef57-4b24-bae7-d64f3051beb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:53.692202 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-856b8772-ef57-4b24-bae7-d64f3051beb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:53.694403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-856b8772-ef57-4b24-bae7-d64f3051beb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:53.695591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-856b8772-ef57-4b24-bae7-d64f3051beb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:53.698637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:53.699085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:53.699860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:53.700269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:53.889999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-856b8772-ef57-4b24-bae7-d64f3051beb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:53.892970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:53.898241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-856b8772-ef57-4b24-bae7-d64f3051beb8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.227922 May 25 16:41:53.899016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 987/1952] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:53 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:53.901701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:53.903267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:53.903604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2eed1a5a-1720-4e05-8e64-ba037447a84a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:53.904201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2eed1a5a-1720-4e05-8e64-ba037447a84a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:53.904417 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0ad630a0-28e4-429a-947e-60085a85738c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.223669 May 25 16:41:53.905099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 966/1953] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:53 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:53.910057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2eed1a5a-1720-4e05-8e64-ba037447a84a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:53.910514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2eed1a5a-1720-4e05-8e64-ba037447a84a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:53.911300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2eed1a5a-1720-4e05-8e64-ba037447a84a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:53.911726 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2eed1a5a-1720-4e05-8e64-ba037447a84a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:53.915436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d2ccebd1-e18a-46ae-91e7-9c450da9e19b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:53.916149 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d2ccebd1-e18a-46ae-91e7-9c450da9e19b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:53.922510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2ccebd1-e18a-46ae-91e7-9c450da9e19b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:53.923038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2ccebd1-e18a-46ae-91e7-9c450da9e19b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:53.923920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2ccebd1-e18a-46ae-91e7-9c450da9e19b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:53.924418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2ccebd1-e18a-46ae-91e7-9c450da9e19b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:53.927208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:41:53.986815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2eed1a5a-1720-4e05-8e64-ba037447a84a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:53.995411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2eed1a5a-1720-4e05-8e64-ba037447a84a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.095473 May 25 16:41:53.996016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 988/1954] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:53 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:54.071797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d2ccebd1-e18a-46ae-91e7-9c450da9e19b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:54.080961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d2ccebd1-e18a-46ae-91e7-9c450da9e19b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.169580 May 25 16:41:54.081534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 967/1955] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:53 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:54.918519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-45350a34-e0a6-49d7-9a8f-26be97e7de91 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:54.919126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-45350a34-e0a6-49d7-9a8f-26be97e7de91 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:54.922659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:54.923332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:54.926106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45350a34-e0a6-49d7-9a8f-26be97e7de91 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:54.926556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45350a34-e0a6-49d7-9a8f-26be97e7de91 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:54.927357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45350a34-e0a6-49d7-9a8f-26be97e7de91 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:54.927765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45350a34-e0a6-49d7-9a8f-26be97e7de91 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:54.930082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:54.930538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:54.931475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:54.931961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:55.120504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-45350a34-e0a6-49d7-9a8f-26be97e7de91 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:55.124568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:55.128777 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-45350a34-e0a6-49d7-9a8f-26be97e7de91 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.216328 May 25 16:41:55.129335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 989/1956] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:54 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:55.133398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:55.133740 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9b1f0d71-aa29-4838-9430-7790a1834ac9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:55.134347 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9b1f0d71-aa29-4838-9430-7790a1834ac9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:55.134990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:55.136262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d2ca3ca6-7351-4ae0-b33c-5a6e61370c00 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.217457 May 25 16:41:55.137021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 968/1957] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:55.140503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b1f0d71-aa29-4838-9430-7790a1834ac9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:55.141236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b1f0d71-aa29-4838-9430-7790a1834ac9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:55.142011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b1f0d71-aa29-4838-9430-7790a1834ac9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:55.142168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fec22ee7-35ba-4d35-8ee1-aaba2f27d50d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:55.142443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b1f0d71-aa29-4838-9430-7790a1834ac9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:55.143111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fec22ee7-35ba-4d35-8ee1-aaba2f27d50d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:55.149755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec22ee7-35ba-4d35-8ee1-aaba2f27d50d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:55.150205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec22ee7-35ba-4d35-8ee1-aaba2f27d50d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:55.151068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec22ee7-35ba-4d35-8ee1-aaba2f27d50d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:55.151536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fec22ee7-35ba-4d35-8ee1-aaba2f27d50d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:55.212954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9b1f0d71-aa29-4838-9430-7790a1834ac9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:55.221069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9b1f0d71-aa29-4838-9430-7790a1834ac9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090752 May 25 16:41:55.221630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 990/1958] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:55 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:55.305133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fec22ee7-35ba-4d35-8ee1-aaba2f27d50d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:55.318369 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fec22ee7-35ba-4d35-8ee1-aaba2f27d50d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.179590 May 25 16:41:55.319191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 969/1959] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:55 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:56.158096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9db9890f-7e17-4032-89a0-fe3d8a93c798 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:56.159136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9db9890f-7e17-4032-89a0-fe3d8a93c798 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:56.160607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:56.161605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:56.168913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9db9890f-7e17-4032-89a0-fe3d8a93c798 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:56.169362 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9db9890f-7e17-4032-89a0-fe3d8a93c798 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:56.170164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9db9890f-7e17-4032-89a0-fe3d8a93c798 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:56.170496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:56.170670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9db9890f-7e17-4032-89a0-fe3d8a93c798 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:56.170950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:56.171748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:56.172184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:56.361093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9db9890f-7e17-4032-89a0-fe3d8a93c798 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:56.364615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:56.369512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9db9890f-7e17-4032-89a0-fe3d8a93c798 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.218581 May 25 16:41:56.370100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 991/1960] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:56 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:56.373610 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:56.374724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-555bed9b-fd7a-4af8-9fc5-3f38a008b10f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:56.375095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:56.375520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-555bed9b-fd7a-4af8-9fc5-3f38a008b10f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:56.376275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-89b23a01-a9f9-406d-8c8d-304f3059d85c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.222136 May 25 16:41:56.377368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 970/1961] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:56 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:56.381820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-555bed9b-fd7a-4af8-9fc5-3f38a008b10f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:56.381976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-19164bf5-65e8-4e78-b723-b58190a8e6dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:56.382278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-555bed9b-fd7a-4af8-9fc5-3f38a008b10f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:56.382413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-19164bf5-65e8-4e78-b723-b58190a8e6dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:56.383047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-555bed9b-fd7a-4af8-9fc5-3f38a008b10f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:56.383520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-555bed9b-fd7a-4af8-9fc5-3f38a008b10f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:56.388452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19164bf5-65e8-4e78-b723-b58190a8e6dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:56.388897 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19164bf5-65e8-4e78-b723-b58190a8e6dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:56.389649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19164bf5-65e8-4e78-b723-b58190a8e6dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:56.390066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19164bf5-65e8-4e78-b723-b58190a8e6dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:56.454715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-555bed9b-fd7a-4af8-9fc5-3f38a008b10f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:56.463427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-555bed9b-fd7a-4af8-9fc5-3f38a008b10f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092190 May 25 16:41:56.464037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 992/1962] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:56 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:56.540839 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-19164bf5-65e8-4e78-b723-b58190a8e6dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:56.549002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-19164bf5-65e8-4e78-b723-b58190a8e6dc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.170435 May 25 16:41:56.549610 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 971/1963] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:56 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:57.392394 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-be85b177-dd11-4c17-be54-b9e1629779e5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:57.393220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-be85b177-dd11-4c17-be54-b9e1629779e5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:57.398607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:57.399605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:57.404929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be85b177-dd11-4c17-be54-b9e1629779e5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:57.405294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be85b177-dd11-4c17-be54-b9e1629779e5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:57.406573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be85b177-dd11-4c17-be54-b9e1629779e5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:57.407332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-be85b177-dd11-4c17-be54-b9e1629779e5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:57.410293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:57.411049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:57.412354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:57.413080 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:57.604998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:57.607729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-be85b177-dd11-4c17-be54-b9e1629779e5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:57.613694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:57.615200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:57.615840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-be85b177-dd11-4c17-be54-b9e1629779e5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.230812 May 25 16:41:57.616412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 993/1964] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:57 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:57.616573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-64a78886-b726-4389-a9ee-006f57f01778 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.224344 May 25 16:41:57.617158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 972/1965] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:57.621825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c15feafa-7cea-4da7-930d-015d32fdf3ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:57.622419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c15feafa-7cea-4da7-930d-015d32fdf3ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:57.625861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6b4ea668-5ff4-4a6a-9fac-a19bd1127539 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:57.626545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6b4ea668-5ff4-4a6a-9fac-a19bd1127539 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:57.628469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c15feafa-7cea-4da7-930d-015d32fdf3ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:57.628923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c15feafa-7cea-4da7-930d-015d32fdf3ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:57.629628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c15feafa-7cea-4da7-930d-015d32fdf3ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:57.630051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c15feafa-7cea-4da7-930d-015d32fdf3ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:57.633357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b4ea668-5ff4-4a6a-9fac-a19bd1127539 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:57.633810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b4ea668-5ff4-4a6a-9fac-a19bd1127539 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:57.634578 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b4ea668-5ff4-4a6a-9fac-a19bd1127539 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:57.635172 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6b4ea668-5ff4-4a6a-9fac-a19bd1127539 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:57.716704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6b4ea668-5ff4-4a6a-9fac-a19bd1127539 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:57.725016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6b4ea668-5ff4-4a6a-9fac-a19bd1127539 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.107642 May 25 16:41:57.725570 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 994/1966] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:57 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:57.781799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c15feafa-7cea-4da7-930d-015d32fdf3ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:57.794423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c15feafa-7cea-4da7-930d-015d32fdf3ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.176357 May 25 16:41:57.795171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 973/1967] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:57 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:58.635609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3edb3e7d-104d-4911-b230-e1318a7a4848 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:58.636321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:58.636812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3edb3e7d-104d-4911-b230-e1318a7a4848 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:58.637214 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:58.645367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3edb3e7d-104d-4911-b230-e1318a7a4848 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:58.645585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:58.645961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3edb3e7d-104d-4911-b230-e1318a7a4848 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:58.646176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:58.646964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3edb3e7d-104d-4911-b230-e1318a7a4848 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:58.647200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:58.647599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3edb3e7d-104d-4911-b230-e1318a7a4848 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:58.647812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:58.841425 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3edb3e7d-104d-4911-b230-e1318a7a4848 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:58.845654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:58.853708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3edb3e7d-104d-4911-b230-e1318a7a4848 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.223180 May 25 16:41:58.854574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 995/1968] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:58 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:58.857761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:58.859180 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8bd47438-18e3-4fbe-9837-62f69895e3d8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:58.859399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:58.859785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8bd47438-18e3-4fbe-9837-62f69895e3d8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:58.860662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-353bfdd1-4bb3-45b2-9a70-6009ffb9b9ce tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.229592 May 25 16:41:58.861488 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 974/1969] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:58 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:41:58.867059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b04549a7-b47e-4dfa-8e0d-1dc5c6d7f196 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:58.867966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b04549a7-b47e-4dfa-8e0d-1dc5c6d7f196 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:58.873134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bd47438-18e3-4fbe-9837-62f69895e3d8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:58.873685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bd47438-18e3-4fbe-9837-62f69895e3d8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:58.874306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b04549a7-b47e-4dfa-8e0d-1dc5c6d7f196 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:58.874565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bd47438-18e3-4fbe-9837-62f69895e3d8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:58.874713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b04549a7-b47e-4dfa-8e0d-1dc5c6d7f196 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:58.874999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bd47438-18e3-4fbe-9837-62f69895e3d8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:58.875520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b04549a7-b47e-4dfa-8e0d-1dc5c6d7f196 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:58.875958 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b04549a7-b47e-4dfa-8e0d-1dc5c6d7f196 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:58.957395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8bd47438-18e3-4fbe-9837-62f69895e3d8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:58.965670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8bd47438-18e3-4fbe-9837-62f69895e3d8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.110157 May 25 16:41:58.966464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 996/1970] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:58 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:59.026176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b04549a7-b47e-4dfa-8e0d-1dc5c6d7f196 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:41:59.034609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b04549a7-b47e-4dfa-8e0d-1dc5c6d7f196 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172320 May 25 16:41:59.035189 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 975/1971] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:58 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:41:59.870919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-df62e085-1b8f-477b-8bef-3651a284765e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:59.871600 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-df62e085-1b8f-477b-8bef-3651a284765e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:59.878605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df62e085-1b8f-477b-8bef-3651a284765e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:59.879066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df62e085-1b8f-477b-8bef-3651a284765e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:59.879838 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df62e085-1b8f-477b-8bef-3651a284765e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:59.880306 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df62e085-1b8f-477b-8bef-3651a284765e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:59.880616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:41:59.881225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:41:59.888443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:59.888900 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:41:59.889685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:41:59.890116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:00.085688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:00.088796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-df62e085-1b8f-477b-8bef-3651a284765e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:00.094329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:00.095870 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:00.096845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-df62e085-1b8f-477b-8bef-3651a284765e tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.229791 May 25 16:42:00.097018 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-accce8f9-21b6-4764-b836-a4c430d17440 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.220408 May 25 16:42:00.097403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 997/1972] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:41:59 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:00.098659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 976/1973] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:41:59 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:00.102517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-28355628-177e-4543-b73f-9a812280319e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:00.103085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-51e2c051-5416-446b-a9db-2c41453d2f66 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:00.103501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-28355628-177e-4543-b73f-9a812280319e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:00.103737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-51e2c051-5416-446b-a9db-2c41453d2f66 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:00.109805 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51e2c051-5416-446b-a9db-2c41453d2f66 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:00.110256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51e2c051-5416-446b-a9db-2c41453d2f66 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:00.110860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-28355628-177e-4543-b73f-9a812280319e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:00.111019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51e2c051-5416-446b-a9db-2c41453d2f66 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:00.111345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-28355628-177e-4543-b73f-9a812280319e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:00.111470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-51e2c051-5416-446b-a9db-2c41453d2f66 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:00.112098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-28355628-177e-4543-b73f-9a812280319e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:00.112526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-28355628-177e-4543-b73f-9a812280319e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:00.199942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-28355628-177e-4543-b73f-9a812280319e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:00.208603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-28355628-177e-4543-b73f-9a812280319e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.110038 May 25 16:42:00.209187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 998/1974] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:00 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:00.265210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-51e2c051-5416-446b-a9db-2c41453d2f66 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:00.273621 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-51e2c051-5416-446b-a9db-2c41453d2f66 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173959 May 25 16:42:00.274188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 977/1975] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:00 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:01.114278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2490e9a7-fbd2-4abc-a820-92491b57296d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:01.114874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2490e9a7-fbd2-4abc-a820-92491b57296d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:01.115956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:01.116568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:01.121764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2490e9a7-fbd2-4abc-a820-92491b57296d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:01.122218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2490e9a7-fbd2-4abc-a820-92491b57296d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:01.122987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2490e9a7-fbd2-4abc-a820-92491b57296d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:01.123333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:01.123475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2490e9a7-fbd2-4abc-a820-92491b57296d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:01.123790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:01.124553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:01.124980 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:01.141383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:01.315061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2490e9a7-fbd2-4abc-a820-92491b57296d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:01.318292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:01.323577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2490e9a7-fbd2-4abc-a820-92491b57296d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.213117 May 25 16:42:01.324093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 999/1976] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:01 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:01.327341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:01.328802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-05ced840-8415-4981-a33c-4162d8dbd4ae tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:01.328985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:01.329410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-05ced840-8415-4981-a33c-4162d8dbd4ae tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:01.330356 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4f80f9a3-6e6b-4e41-bc70-3e851669c6c4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.218326 May 25 16:42:01.331128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 978/1977] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:01 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:01.335423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-05ced840-8415-4981-a33c-4162d8dbd4ae tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:01.336092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-05ced840-8415-4981-a33c-4162d8dbd4ae tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:01.337088 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-05ced840-8415-4981-a33c-4162d8dbd4ae tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:01.337530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-05ced840-8415-4981-a33c-4162d8dbd4ae tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:01.338901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-84486936-0187-496e-92c7-e9e8853587f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:01.339532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-84486936-0187-496e-92c7-e9e8853587f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:01.346015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-84486936-0187-496e-92c7-e9e8853587f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:01.346467 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-84486936-0187-496e-92c7-e9e8853587f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:01.347254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-84486936-0187-496e-92c7-e9e8853587f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:01.347690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-84486936-0187-496e-92c7-e9e8853587f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:01.420018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-05ced840-8415-4981-a33c-4162d8dbd4ae tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:01.428472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-05ced840-8415-4981-a33c-4162d8dbd4ae tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.103166 May 25 16:42:01.429022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1000/1978] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:01 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:01.492691 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-84486936-0187-496e-92c7-e9e8853587f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:01.501152 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-84486936-0187-496e-92c7-e9e8853587f4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.165898 May 25 16:42:01.501738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 979/1979] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:01 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:02.343997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7d25ffd7-91e4-4240-a567-cafa35246fe6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:02.344784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7d25ffd7-91e4-4240-a567-cafa35246fe6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:02.350644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:02.351449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:02.353908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d25ffd7-91e4-4240-a567-cafa35246fe6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:02.354580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d25ffd7-91e4-4240-a567-cafa35246fe6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:02.355642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d25ffd7-91e4-4240-a567-cafa35246fe6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:02.356257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7d25ffd7-91e4-4240-a567-cafa35246fe6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:02.360849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:02.361442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:02.362435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:02.363076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:02.555000 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7d25ffd7-91e4-4240-a567-cafa35246fe6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:02.558294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:02.564033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7d25ffd7-91e4-4240-a567-cafa35246fe6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.225406 May 25 16:42:02.564675 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1001/1980] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:02 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:02.568210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:02.569147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2bc9fede-f78f-41bf-bd84-319ad17660f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:02.569718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:02.569871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2bc9fede-f78f-41bf-bd84-319ad17660f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:02.570880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f2539f63-260c-4a40-aafe-7c7a5828427e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.225562 May 25 16:42:02.571459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 980/1981] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:02 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:02.576230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2bc9fede-f78f-41bf-bd84-319ad17660f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:02.576674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2bc9fede-f78f-41bf-bd84-319ad17660f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:02.577166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b3300367-ec8b-4b63-bf58-22556cc93d86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:02.577397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2bc9fede-f78f-41bf-bd84-319ad17660f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:02.577851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2bc9fede-f78f-41bf-bd84-319ad17660f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:02.578005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b3300367-ec8b-4b63-bf58-22556cc93d86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:02.584265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3300367-ec8b-4b63-bf58-22556cc93d86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:02.584719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3300367-ec8b-4b63-bf58-22556cc93d86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:02.585485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3300367-ec8b-4b63-bf58-22556cc93d86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:02.585918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b3300367-ec8b-4b63-bf58-22556cc93d86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:02.660426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2bc9fede-f78f-41bf-bd84-319ad17660f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:02.670586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2bc9fede-f78f-41bf-bd84-319ad17660f0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.104956 May 25 16:42:02.671182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1002/1982] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:02 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:02.733147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b3300367-ec8b-4b63-bf58-22556cc93d86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:02.742341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b3300367-ec8b-4b63-bf58-22556cc93d86 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.169691 May 25 16:42:02.742901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 981/1983] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:02 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:03.585298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-99dda7ff-4bf0-4b41-8fbd-24ad7ec6c536 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:03.586327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-99dda7ff-4bf0-4b41-8fbd-24ad7ec6c536 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:03.593465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:03.594466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:03.597278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-99dda7ff-4bf0-4b41-8fbd-24ad7ec6c536 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:03.598019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-99dda7ff-4bf0-4b41-8fbd-24ad7ec6c536 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:03.599314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-99dda7ff-4bf0-4b41-8fbd-24ad7ec6c536 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:03.600033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-99dda7ff-4bf0-4b41-8fbd-24ad7ec6c536 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:03.604719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:03.605270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:03.606103 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:03.606600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:03.794761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-99dda7ff-4bf0-4b41-8fbd-24ad7ec6c536 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:03.798155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:03.802983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-99dda7ff-4bf0-4b41-8fbd-24ad7ec6c536 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.224268 May 25 16:42:03.803596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1003/1984] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:03 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:03.807423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:03.808175 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c47b2d65-0a35-46d8-8dbf-4311bd18ab97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:03.808888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c47b2d65-0a35-46d8-8dbf-4311bd18ab97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:03.809037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:03.810108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c315ab9d-ad74-495c-a12e-591c0c0db324 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.223066 May 25 16:42:03.810802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 982/1985] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:03 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:03.814888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c47b2d65-0a35-46d8-8dbf-4311bd18ab97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:03.815614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c47b2d65-0a35-46d8-8dbf-4311bd18ab97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:03.816113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-79619f81-fa45-477b-8ee2-e2e7a620dda1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:03.816719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-79619f81-fa45-477b-8ee2-e2e7a620dda1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:03.817069 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c47b2d65-0a35-46d8-8dbf-4311bd18ab97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:03.817587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c47b2d65-0a35-46d8-8dbf-4311bd18ab97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:03.822753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-79619f81-fa45-477b-8ee2-e2e7a620dda1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:03.823403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-79619f81-fa45-477b-8ee2-e2e7a620dda1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:03.824307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-79619f81-fa45-477b-8ee2-e2e7a620dda1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:03.824745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-79619f81-fa45-477b-8ee2-e2e7a620dda1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:03.886286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c47b2d65-0a35-46d8-8dbf-4311bd18ab97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:03.894289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c47b2d65-0a35-46d8-8dbf-4311bd18ab97 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089765 May 25 16:42:03.894839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1004/1986] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:03 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:03.971623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-79619f81-fa45-477b-8ee2-e2e7a620dda1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:03.979627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-79619f81-fa45-477b-8ee2-e2e7a620dda1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.167168 May 25 16:42:03.980264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 983/1987] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:03 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:04.824599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-85bce136-2d5a-4a3a-b07c-20d8f910d78c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:04.825647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-85bce136-2d5a-4a3a-b07c-20d8f910d78c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:04.833644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:04.834803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:04.836656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85bce136-2d5a-4a3a-b07c-20d8f910d78c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:04.837422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85bce136-2d5a-4a3a-b07c-20d8f910d78c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:04.838527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85bce136-2d5a-4a3a-b07c-20d8f910d78c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:04.838962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-85bce136-2d5a-4a3a-b07c-20d8f910d78c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:04.841643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:04.843071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:04.843640 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:04.844560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:04.844991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:05.268467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-85bce136-2d5a-4a3a-b07c-20d8f910d78c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:05.271404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:05.276895 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-85bce136-2d5a-4a3a-b07c-20d8f910d78c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.458935 May 25 16:42:05.277446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1005/1988] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:04 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:05.280175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:05.281683 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:05.281908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-354f8044-cf11-4dfc-b2cb-a46d6ded2c94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:05.282498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-354f8044-cf11-4dfc-b2cb-a46d6ded2c94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:05.282849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-015d6fbe-a3c1-4cee-ac0d-8754cab0759e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.456165 May 25 16:42:05.287107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 984/1989] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:04 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:05.288172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a1814665-f0e4-4124-8995-98a3c3f62f5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:05.288412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-354f8044-cf11-4dfc-b2cb-a46d6ded2c94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:05.288781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a1814665-f0e4-4124-8995-98a3c3f62f5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:05.288947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-354f8044-cf11-4dfc-b2cb-a46d6ded2c94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:05.289890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-354f8044-cf11-4dfc-b2cb-a46d6ded2c94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:05.290346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-354f8044-cf11-4dfc-b2cb-a46d6ded2c94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:05.295632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1814665-f0e4-4124-8995-98a3c3f62f5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:05.296107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1814665-f0e4-4124-8995-98a3c3f62f5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:05.296876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1814665-f0e4-4124-8995-98a3c3f62f5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:05.297297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a1814665-f0e4-4124-8995-98a3c3f62f5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:05.361204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-354f8044-cf11-4dfc-b2cb-a46d6ded2c94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:05.369609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-354f8044-cf11-4dfc-b2cb-a46d6ded2c94 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091179 May 25 16:42:05.370216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1006/1990] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:05 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:05.444285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a1814665-f0e4-4124-8995-98a3c3f62f5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:05.456496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a1814665-f0e4-4124-8995-98a3c3f62f5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.171419 May 25 16:42:05.457386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 985/1991] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:05 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:06.299504 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0776ea99-90cb-4a61-bd9e-da2e1f7aeb80 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:06.300535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0776ea99-90cb-4a61-bd9e-da2e1f7aeb80 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:06.305676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:06.306742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:06.309976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0776ea99-90cb-4a61-bd9e-da2e1f7aeb80 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:06.310421 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0776ea99-90cb-4a61-bd9e-da2e1f7aeb80 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:06.311213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0776ea99-90cb-4a61-bd9e-da2e1f7aeb80 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:06.311735 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0776ea99-90cb-4a61-bd9e-da2e1f7aeb80 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:06.313039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:06.313496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:06.314297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:06.314739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:06.504609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0776ea99-90cb-4a61-bd9e-da2e1f7aeb80 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:06.507865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:06.513435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0776ea99-90cb-4a61-bd9e-da2e1f7aeb80 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221036 May 25 16:42:06.514043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1007/1992] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:06 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:06.517370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:06.518640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ac910f84-bbda-45d4-bedf-e14cf202f048 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:06.518886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:06.519266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ac910f84-bbda-45d4-bedf-e14cf202f048 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:06.520081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c7cd4a46-5af5-4725-8a07-ca8e8c24b2ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.221129 May 25 16:42:06.520620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 986/1993] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:06 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:06.525117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a8654bf2-8d80-4baf-9a2c-f14fa13ed1f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:06.525466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac910f84-bbda-45d4-bedf-e14cf202f048 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:06.525687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a8654bf2-8d80-4baf-9a2c-f14fa13ed1f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:06.525958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac910f84-bbda-45d4-bedf-e14cf202f048 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:06.526677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac910f84-bbda-45d4-bedf-e14cf202f048 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:06.527191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ac910f84-bbda-45d4-bedf-e14cf202f048 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:06.532008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8654bf2-8d80-4baf-9a2c-f14fa13ed1f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:06.532460 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8654bf2-8d80-4baf-9a2c-f14fa13ed1f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:06.533220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8654bf2-8d80-4baf-9a2c-f14fa13ed1f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:06.533652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8654bf2-8d80-4baf-9a2c-f14fa13ed1f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:06.614354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ac910f84-bbda-45d4-bedf-e14cf202f048 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:06.622429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ac910f84-bbda-45d4-bedf-e14cf202f048 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.107307 May 25 16:42:06.622978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1008/1994] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:06 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:06.680789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a8654bf2-8d80-4baf-9a2c-f14fa13ed1f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:06.689086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a8654bf2-8d80-4baf-9a2c-f14fa13ed1f6 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.167336 May 25 16:42:06.689661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 987/1995] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:06 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:07.534435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f3e4d561-249f-4085-9834-38bce347fdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:07.535526 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f3e4d561-249f-4085-9834-38bce347fdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:07.543054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:07.544147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:07.548251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3e4d561-249f-4085-9834-38bce347fdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:07.549166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3e4d561-249f-4085-9834-38bce347fdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:07.550607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3e4d561-249f-4085-9834-38bce347fdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:07.551385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3e4d561-249f-4085-9834-38bce347fdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:07.555034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:07.555847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:07.556996 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:07.557433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:07.744191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f3e4d561-249f-4085-9834-38bce347fdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:07.748279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:07.752581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f3e4d561-249f-4085-9834-38bce347fdea tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.224769 May 25 16:42:07.753160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1009/1996] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:07 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:07.757361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:07.757534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9b7b4e4d-209b-496f-8133-e3da36c067aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:07.758125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9b7b4e4d-209b-496f-8133-e3da36c067aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:07.759246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:07.760692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-76fee51b-25eb-40ac-b5cf-da18f1692354 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.225106 May 25 16:42:07.762190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 988/1997] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:07 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:07.764356 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b7b4e4d-209b-496f-8133-e3da36c067aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:07.764801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b7b4e4d-209b-496f-8133-e3da36c067aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:07.765564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b7b4e4d-209b-496f-8133-e3da36c067aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:07.766277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b7b4e4d-209b-496f-8133-e3da36c067aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:07.766647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-98348e93-5a52-4958-9e03-634f8f44e614 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:07.767345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-98348e93-5a52-4958-9e03-634f8f44e614 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:07.773468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98348e93-5a52-4958-9e03-634f8f44e614 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:07.773916 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98348e93-5a52-4958-9e03-634f8f44e614 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:07.774669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98348e93-5a52-4958-9e03-634f8f44e614 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:07.775096 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98348e93-5a52-4958-9e03-634f8f44e614 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:07.835852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9b7b4e4d-209b-496f-8133-e3da36c067aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:07.844155 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9b7b4e4d-209b-496f-8133-e3da36c067aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090020 May 25 16:42:07.844741 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1010/1998] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:07 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:07.923702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-98348e93-5a52-4958-9e03-634f8f44e614 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:07.931690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-98348e93-5a52-4958-9e03-634f8f44e614 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.168922 May 25 16:42:07.932382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 989/1999] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:07 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:08.773916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f28de93f-e0a3-4ff4-8d96-49e82ae7dbef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:08.774993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f28de93f-e0a3-4ff4-8d96-49e82ae7dbef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:08.783702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:08.784699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:08.786108 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f28de93f-e0a3-4ff4-8d96-49e82ae7dbef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:08.786876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f28de93f-e0a3-4ff4-8d96-49e82ae7dbef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:08.788276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f28de93f-e0a3-4ff4-8d96-49e82ae7dbef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:08.788999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f28de93f-e0a3-4ff4-8d96-49e82ae7dbef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:08.794143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:08.794616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:08.795405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:08.795827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:08.981499 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:08.985449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f28de93f-e0a3-4ff4-8d96-49e82ae7dbef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:08.990179 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:08.991725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:08.992880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c755632e-a3d6-46b6-be66-e1a7f32481a2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.215711 May 25 16:42:08.993411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 990/2000] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:08 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:08.995268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f28de93f-e0a3-4ff4-8d96-49e82ae7dbef tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.227893 May 25 16:42:08.996013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1011/2001] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:08 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:08.997850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-555fc112-c2f6-4928-9de3-94d3754ab77d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:08.998452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-555fc112-c2f6-4928-9de3-94d3754ab77d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:09.001444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a826eb5b-6a00-413b-ab30-15c6541401ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:09.002055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a826eb5b-6a00-413b-ab30-15c6541401ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:09.004674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-555fc112-c2f6-4928-9de3-94d3754ab77d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:09.004984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-555fc112-c2f6-4928-9de3-94d3754ab77d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:09.005949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-555fc112-c2f6-4928-9de3-94d3754ab77d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:09.006563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-555fc112-c2f6-4928-9de3-94d3754ab77d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:09.009134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a826eb5b-6a00-413b-ab30-15c6541401ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:09.009653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a826eb5b-6a00-413b-ab30-15c6541401ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:09.011582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a826eb5b-6a00-413b-ab30-15c6541401ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:09.012010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a826eb5b-6a00-413b-ab30-15c6541401ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:09.076809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-555fc112-c2f6-4928-9de3-94d3754ab77d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:09.085096 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-555fc112-c2f6-4928-9de3-94d3754ab77d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090721 May 25 16:42:09.085693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 991/2002] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:08 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:09.159537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a826eb5b-6a00-413b-ab30-15c6541401ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:09.168396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a826eb5b-6a00-413b-ab30-15c6541401ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.171598 May 25 16:42:09.168972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1012/2003] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:08 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:10.017562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-66858c3c-88c8-4c90-81a7-11845d4b282a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:10.018498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-66858c3c-88c8-4c90-81a7-11845d4b282a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:10.025139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:10.025887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:10.028233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66858c3c-88c8-4c90-81a7-11845d4b282a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:10.028686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66858c3c-88c8-4c90-81a7-11845d4b282a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:10.029482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66858c3c-88c8-4c90-81a7-11845d4b282a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:10.029922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66858c3c-88c8-4c90-81a7-11845d4b282a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:10.032775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:10.033231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:10.034001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:10.034425 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:10.222292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-66858c3c-88c8-4c90-81a7-11845d4b282a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:10.226766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:10.232539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-66858c3c-88c8-4c90-81a7-11845d4b282a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221635 May 25 16:42:10.233325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 992/2004] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:10 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:10.236020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:10.237525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:10.237703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-70a7cbb0-359a-48bb-b980-79a899adaf6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:10.238207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-70a7cbb0-359a-48bb-b980-79a899adaf6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:10.238833 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fd9de520-705d-44db-bae9-b8f297ae2997 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.220627 May 25 16:42:10.247338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1013/2005] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:10 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:10.247338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3dc4f1ab-13fa-4666-b2e6-ae72f05576ad tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:10.247338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3dc4f1ab-13fa-4666-b2e6-ae72f05576ad tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:10.247718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70a7cbb0-359a-48bb-b980-79a899adaf6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:10.247718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70a7cbb0-359a-48bb-b980-79a899adaf6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:10.247718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70a7cbb0-359a-48bb-b980-79a899adaf6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:10.247718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-70a7cbb0-359a-48bb-b980-79a899adaf6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:10.250429 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3dc4f1ab-13fa-4666-b2e6-ae72f05576ad tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:10.250884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3dc4f1ab-13fa-4666-b2e6-ae72f05576ad tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:10.251676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3dc4f1ab-13fa-4666-b2e6-ae72f05576ad tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:10.252114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3dc4f1ab-13fa-4666-b2e6-ae72f05576ad tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:10.331228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-70a7cbb0-359a-48bb-b980-79a899adaf6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:10.339426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-70a7cbb0-359a-48bb-b980-79a899adaf6f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.105289 May 25 16:42:10.339945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 993/2006] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:10 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:10.400410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3dc4f1ab-13fa-4666-b2e6-ae72f05576ad tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:10.409117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3dc4f1ab-13fa-4666-b2e6-ae72f05576ad tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.168747 May 25 16:42:10.409688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1014/2007] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:10 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:11.254871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-44d04755-46ca-4917-bdbb-e9cdc378b52a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:11.255860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-44d04755-46ca-4917-bdbb-e9cdc378b52a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:11.260528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:11.261110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:11.264306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-44d04755-46ca-4917-bdbb-e9cdc378b52a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:11.264785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-44d04755-46ca-4917-bdbb-e9cdc378b52a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:11.265555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-44d04755-46ca-4917-bdbb-e9cdc378b52a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:11.265993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-44d04755-46ca-4917-bdbb-e9cdc378b52a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:11.267820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:11.268276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:11.269037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:11.269471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:11.481146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-44d04755-46ca-4917-bdbb-e9cdc378b52a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:11.486273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:11.492985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-44d04755-46ca-4917-bdbb-e9cdc378b52a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.244467 May 25 16:42:11.493979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 994/2008] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:11 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:11.499005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0d118791-177d-4783-ac25-b3e35ce62b92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:11.499496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:11.499641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0d118791-177d-4783-ac25-b3e35ce62b92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:11.501144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:11.502310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bcf3162f-ec3c-42bf-856b-683514781419 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.247945 May 25 16:42:11.502441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=98 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:11.512882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1015/2009] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:11 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:11.513072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d118791-177d-4783-ac25-b3e35ce62b92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:11.513072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d118791-177d-4783-ac25-b3e35ce62b92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:11.513072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d118791-177d-4783-ac25-b3e35ce62b92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:11.513353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-273cf09b-e822-421f-bf87-1146a95272f1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:11.513440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d118791-177d-4783-ac25-b3e35ce62b92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:11.513525 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-273cf09b-e822-421f-bf87-1146a95272f1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:11.513622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:11.515561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-273cf09b-e822-421f-bf87-1146a95272f1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:11.516007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-273cf09b-e822-421f-bf87-1146a95272f1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:11.516760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-273cf09b-e822-421f-bf87-1146a95272f1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:11.517190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-273cf09b-e822-421f-bf87-1146a95272f1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:11.580047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0d118791-177d-4783-ac25-b3e35ce62b92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:11.588332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0d118791-177d-4783-ac25-b3e35ce62b92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093314 May 25 16:42:11.588873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 995/2010] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:11 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:11.665652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-273cf09b-e822-421f-bf87-1146a95272f1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:11.674414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-273cf09b-e822-421f-bf87-1146a95272f1 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.169945 May 25 16:42:11.674948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1016/2011] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:11 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:12.513789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fa8e5ff7-6341-42ec-9a8e-77d868a892be tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:12.514497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fa8e5ff7-6341-42ec-9a8e-77d868a892be tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:12.520975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:12.521440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa8e5ff7-6341-42ec-9a8e-77d868a892be tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:12.521583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:12.521819 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa8e5ff7-6341-42ec-9a8e-77d868a892be tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:12.522572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa8e5ff7-6341-42ec-9a8e-77d868a892be tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:12.522995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fa8e5ff7-6341-42ec-9a8e-77d868a892be tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:12.528187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:12.528644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:12.529407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:12.529842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:12.720424 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fa8e5ff7-6341-42ec-9a8e-77d868a892be tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:12.724431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:12.729160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fa8e5ff7-6341-42ec-9a8e-77d868a892be tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221258 May 25 16:42:12.729751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 996/2012] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:12 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:12.733386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:12.734169 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f0f42910-c4b9-4a8f-a6b3-454eacce1e42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:12.734773 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f0f42910-c4b9-4a8f-a6b3-454eacce1e42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:12.734982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:12.736127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-03e7bf56-5df1-4f21-b5dc-6691cc21ed33 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.219152 May 25 16:42:12.736879 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1017/2013] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:12 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:12.741181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ff81b494-433d-4577-be8b-66367bda0c9a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:12.741779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ff81b494-433d-4577-be8b-66367bda0c9a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:12.748064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ff81b494-433d-4577-be8b-66367bda0c9a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:12.748508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ff81b494-433d-4577-be8b-66367bda0c9a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:12.749270 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ff81b494-433d-4577-be8b-66367bda0c9a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:12.749709 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ff81b494-433d-4577-be8b-66367bda0c9a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:12.750665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f0f42910-c4b9-4a8f-a6b3-454eacce1e42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:12.751127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f0f42910-c4b9-4a8f-a6b3-454eacce1e42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:12.752013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f0f42910-c4b9-4a8f-a6b3-454eacce1e42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:12.752350 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f0f42910-c4b9-4a8f-a6b3-454eacce1e42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:12.838969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f0f42910-c4b9-4a8f-a6b3-454eacce1e42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:12.848453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f0f42910-c4b9-4a8f-a6b3-454eacce1e42 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.117679 May 25 16:42:12.849075 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 997/2014] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:12 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:12.907195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ff81b494-433d-4577-be8b-66367bda0c9a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:12.919909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ff81b494-433d-4577-be8b-66367bda0c9a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.181765 May 25 16:42:12.920658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1018/2015] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:12 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:13.641604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:42:13.642697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:42:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:42:13.749099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ee383f72-88c4-4135-bf45-252248fbaa76 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:13.750042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ee383f72-88c4-4135-bf45-252248fbaa76 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:13.756015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:13.756774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:13.758929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee383f72-88c4-4135-bf45-252248fbaa76 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:13.759540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee383f72-88c4-4135-bf45-252248fbaa76 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:13.760513 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee383f72-88c4-4135-bf45-252248fbaa76 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:13.761014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee383f72-88c4-4135-bf45-252248fbaa76 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:13.765278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:13.765851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:13.766796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:13.767377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:13.959752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ee383f72-88c4-4135-bf45-252248fbaa76 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:13.962824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:13.967929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ee383f72-88c4-4135-bf45-252248fbaa76 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.223810 May 25 16:42:13.968544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 998/2016] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:13 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:13.971857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:13.973130 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-081c0dd4-83f1-47aa-94a6-c59349c54053 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:13.973310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:13.973721 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-081c0dd4-83f1-47aa-94a6-c59349c54053 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:13.974505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-47d64275-2d71-4395-9403-4f24ccfa89c7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.223325 May 25 16:42:13.975339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1019/2017] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:13 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:13.980830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-081c0dd4-83f1-47aa-94a6-c59349c54053 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:13.981269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-081c0dd4-83f1-47aa-94a6-c59349c54053 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:13.982049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-081c0dd4-83f1-47aa-94a6-c59349c54053 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:13.982483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-081c0dd4-83f1-47aa-94a6-c59349c54053 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:13.983537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3678650e-f3c5-48ac-9ccf-8726049f1529 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:13.984126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3678650e-f3c5-48ac-9ccf-8726049f1529 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:13.990095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3678650e-f3c5-48ac-9ccf-8726049f1529 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:13.990532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3678650e-f3c5-48ac-9ccf-8726049f1529 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:13.991305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3678650e-f3c5-48ac-9ccf-8726049f1529 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:13.991735 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3678650e-f3c5-48ac-9ccf-8726049f1529 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:14.050432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-081c0dd4-83f1-47aa-94a6-c59349c54053 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:14.058822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-081c0dd4-83f1-47aa-94a6-c59349c54053 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089360 May 25 16:42:14.059427 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 999/2018] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:13 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:14.134658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3678650e-f3c5-48ac-9ccf-8726049f1529 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:14.142914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3678650e-f3c5-48ac-9ccf-8726049f1529 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162897 May 25 16:42:14.143488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1020/2019] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:13 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:14.986438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b4df4a4b-e7c5-4f08-80b4-129f000ed828 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:14.987113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b4df4a4b-e7c5-4f08-80b4-129f000ed828 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:14.993583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:14.994334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:14.995448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4df4a4b-e7c5-4f08-80b4-129f000ed828 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:14.995981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4df4a4b-e7c5-4f08-80b4-129f000ed828 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:14.996879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4df4a4b-e7c5-4f08-80b4-129f000ed828 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:14.997385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4df4a4b-e7c5-4f08-80b4-129f000ed828 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:15.002155 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:15.002694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:15.003615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:15.004340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:15.193208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b4df4a4b-e7c5-4f08-80b4-129f000ed828 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:15.196289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:15.198523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:15.201922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b4df4a4b-e7c5-4f08-80b4-129f000ed828 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.220078 May 25 16:42:15.202731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1000/2020] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:14 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:15.207173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dcddbbb9-a4d5-487c-91e9-92f11f7d2984 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:15.207786 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dcddbbb9-a4d5-487c-91e9-92f11f7d2984 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:15.213852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:15.214247 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcddbbb9-a4d5-487c-91e9-92f11f7d2984 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:15.214705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcddbbb9-a4d5-487c-91e9-92f11f7d2984 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:15.215390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:15.215518 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcddbbb9-a4d5-487c-91e9-92f11f7d2984 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:15.215947 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcddbbb9-a4d5-487c-91e9-92f11f7d2984 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:15.216575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-68ce244f-5772-4545-80b9-4578085a445c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.227574 May 25 16:42:15.217111 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1021/2021] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:14 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:15.221511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-afce1771-a48b-492d-9e31-71aa03a1acb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:15.222111 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-afce1771-a48b-492d-9e31-71aa03a1acb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:15.231937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-afce1771-a48b-492d-9e31-71aa03a1acb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:15.232390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-afce1771-a48b-492d-9e31-71aa03a1acb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:15.233191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-afce1771-a48b-492d-9e31-71aa03a1acb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:15.233630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-afce1771-a48b-492d-9e31-71aa03a1acb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:15.282898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dcddbbb9-a4d5-487c-91e9-92f11f7d2984 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:15.291747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dcddbbb9-a4d5-487c-91e9-92f11f7d2984 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088153 May 25 16:42:15.292454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1001/2022] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:15 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:15.380076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-afce1771-a48b-492d-9e31-71aa03a1acb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:15.390442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-afce1771-a48b-492d-9e31-71aa03a1acb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172362 May 25 16:42:15.391043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1022/2023] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:15 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:16.223316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2d39e6ca-b2bc-4c52-b8ec-3ec0e5ca2301 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:16.224304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2d39e6ca-b2bc-4c52-b8ec-3ec0e5ca2301 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:16.232842 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d39e6ca-b2bc-4c52-b8ec-3ec0e5ca2301 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:16.233290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d39e6ca-b2bc-4c52-b8ec-3ec0e5ca2301 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:16.234042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d39e6ca-b2bc-4c52-b8ec-3ec0e5ca2301 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:16.234454 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:16.234917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2d39e6ca-b2bc-4c52-b8ec-3ec0e5ca2301 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:16.235041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:16.241932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:16.242382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:16.243131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:16.243624 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:16.420172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2d39e6ca-b2bc-4c52-b8ec-3ec0e5ca2301 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:16.428419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2d39e6ca-b2bc-4c52-b8ec-3ec0e5ca2301 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.211546 May 25 16:42:16.428993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1002/2024] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:16 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:16.430889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:16.433588 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d059724d-96cf-4a34-bf80-385fb1546285 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:16.434191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d059724d-96cf-4a34-bf80-385fb1546285 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:16.440533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:16.440713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d059724d-96cf-4a34-bf80-385fb1546285 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:16.440982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d059724d-96cf-4a34-bf80-385fb1546285 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:16.441746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d059724d-96cf-4a34-bf80-385fb1546285 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:16.442031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:16.442160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d059724d-96cf-4a34-bf80-385fb1546285 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:16.443244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1c24ec0f-d41b-4a06-9ed5-a4df9fad82de tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.212773 May 25 16:42:16.443814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1023/2025] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:16 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:16.448268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fd45ec06-209a-4e53-98a6-624e0ed80abc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:16.448997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fd45ec06-209a-4e53-98a6-624e0ed80abc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:16.454896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd45ec06-209a-4e53-98a6-624e0ed80abc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:16.455387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd45ec06-209a-4e53-98a6-624e0ed80abc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:16.456164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd45ec06-209a-4e53-98a6-624e0ed80abc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:16.456592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fd45ec06-209a-4e53-98a6-624e0ed80abc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:16.505426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d059724d-96cf-4a34-bf80-385fb1546285 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:16.513717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d059724d-96cf-4a34-bf80-385fb1546285 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083695 May 25 16:42:16.514297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1003/2026] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:16 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:16.602769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fd45ec06-209a-4e53-98a6-624e0ed80abc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:16.615670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fd45ec06-209a-4e53-98a6-624e0ed80abc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.170588 May 25 16:42:16.616502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1024/2027] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:16 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:17.449967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-949c1a2b-444f-445a-8b79-4ea319570335 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:17.450841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-949c1a2b-444f-445a-8b79-4ea319570335 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:17.461863 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-949c1a2b-444f-445a-8b79-4ea319570335 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:17.462697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-949c1a2b-444f-445a-8b79-4ea319570335 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:17.463522 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-949c1a2b-444f-445a-8b79-4ea319570335 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:17.463954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-949c1a2b-444f-445a-8b79-4ea319570335 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:17.464779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:17.465407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:17.472224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:17.472667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:17.473404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:17.473834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:17.669568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-949c1a2b-444f-445a-8b79-4ea319570335 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:17.672928 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:17.677979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-949c1a2b-444f-445a-8b79-4ea319570335 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.234607 May 25 16:42:17.678620 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1004/2028] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:17 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:17.683146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-30f1ed5d-716a-45a1-8003-16f412d9c23d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:17.683803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-30f1ed5d-716a-45a1-8003-16f412d9c23d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:17.686694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:17.688228 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:17.689402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6197b85b-e045-46b1-826e-a022e13d6a05 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.230395 May 25 16:42:17.689959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1025/2029] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:17 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:17.690219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30f1ed5d-716a-45a1-8003-16f412d9c23d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:17.690433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30f1ed5d-716a-45a1-8003-16f412d9c23d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:17.691303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30f1ed5d-716a-45a1-8003-16f412d9c23d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:17.691727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-30f1ed5d-716a-45a1-8003-16f412d9c23d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:17.694583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a518e13c-3d81-4a90-bcb5-1cb743d386cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:17.695201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a518e13c-3d81-4a90-bcb5-1cb743d386cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:17.697110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=98 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:17.705817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a518e13c-3d81-4a90-bcb5-1cb743d386cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:17.706211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a518e13c-3d81-4a90-bcb5-1cb743d386cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:17.706986 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a518e13c-3d81-4a90-bcb5-1cb743d386cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:17.707477 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a518e13c-3d81-4a90-bcb5-1cb743d386cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:17.766974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-30f1ed5d-716a-45a1-8003-16f412d9c23d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:17.775670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-30f1ed5d-716a-45a1-8003-16f412d9c23d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.095904 May 25 16:42:17.776281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1005/2030] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:17 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:17.860267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a518e13c-3d81-4a90-bcb5-1cb743d386cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:17.868809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a518e13c-3d81-4a90-bcb5-1cb743d386cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.177750 May 25 16:42:17.869495 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1026/2031] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:17 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:18.699832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-97bd33ac-a103-40f9-b529-819c609a1a8d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:18.700692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-97bd33ac-a103-40f9-b529-819c609a1a8d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:18.711951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97bd33ac-a103-40f9-b529-819c609a1a8d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:18.712693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:18.713506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97bd33ac-a103-40f9-b529-819c609a1a8d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:18.713681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:18.714015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97bd33ac-a103-40f9-b529-819c609a1a8d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:18.714734 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97bd33ac-a103-40f9-b529-819c609a1a8d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:18.724490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:18.725236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:18.726494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:18.727220 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:18.922448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-97bd33ac-a103-40f9-b529-819c609a1a8d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:18.926055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:18.930531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-97bd33ac-a103-40f9-b529-819c609a1a8d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.237414 May 25 16:42:18.931108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1006/2032] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:18 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:18.934943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:18.935755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e8b7b95b-b965-4bfd-86d1-654da80dc967 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:18.936350 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e8b7b95b-b965-4bfd-86d1-654da80dc967 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:18.936507 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:18.937632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c7dcf5b7-e12c-41e4-ab8c-c5956a0ea965 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.231697 May 25 16:42:18.938502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1027/2033] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:18.943118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9b5325a3-1e12-413b-b7c6-f25c85ee5f5c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:18.943757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9b5325a3-1e12-413b-b7c6-f25c85ee5f5c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:18.947448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8b7b95b-b965-4bfd-86d1-654da80dc967 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:18.947963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8b7b95b-b965-4bfd-86d1-654da80dc967 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:18.948720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8b7b95b-b965-4bfd-86d1-654da80dc967 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:18.949157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8b7b95b-b965-4bfd-86d1-654da80dc967 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:18.953194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b5325a3-1e12-413b-b7c6-f25c85ee5f5c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:18.953581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b5325a3-1e12-413b-b7c6-f25c85ee5f5c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:18.954340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b5325a3-1e12-413b-b7c6-f25c85ee5f5c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:18.954775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9b5325a3-1e12-413b-b7c6-f25c85ee5f5c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:19.031913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e8b7b95b-b965-4bfd-86d1-654da80dc967 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:19.044154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e8b7b95b-b965-4bfd-86d1-654da80dc967 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.111713 May 25 16:42:19.045230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1007/2034] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:18 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:19.116673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9b5325a3-1e12-413b-b7c6-f25c85ee5f5c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:19.125437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9b5325a3-1e12-413b-b7c6-f25c85ee5f5c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.185683 May 25 16:42:19.126011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1028/2035] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:18 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:19.950608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3943219d-3218-4f9f-9bea-0bfb730e00a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:19.951390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3943219d-3218-4f9f-9bea-0bfb730e00a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:19.956301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:19.956983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:19.958596 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3943219d-3218-4f9f-9bea-0bfb730e00a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:19.958908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3943219d-3218-4f9f-9bea-0bfb730e00a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:19.959742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3943219d-3218-4f9f-9bea-0bfb730e00a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:19.960533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3943219d-3218-4f9f-9bea-0bfb730e00a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:19.964034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:19.964534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:19.965341 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:19.965773 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:20.154394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3943219d-3218-4f9f-9bea-0bfb730e00a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:20.157184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:20.162824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3943219d-3218-4f9f-9bea-0bfb730e00a7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217320 May 25 16:42:20.163559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1008/2036] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:19 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:20.166489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:20.168013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:20.168511 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4dfa9a15-5bb5-44de-b816-9e13bb5443ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:20.169116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4dfa9a15-5bb5-44de-b816-9e13bb5443ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:20.169278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d23e975d-1552-4c4d-904e-5b4fd51b3e02 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.216991 May 25 16:42:20.169853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1029/2037] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:19 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:20.174175 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b07063a2-e853-482b-a470-ebc7000ffdae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:20.174772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b07063a2-e853-482b-a470-ebc7000ffdae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:20.178566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4dfa9a15-5bb5-44de-b816-9e13bb5443ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:20.179026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4dfa9a15-5bb5-44de-b816-9e13bb5443ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:20.179849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4dfa9a15-5bb5-44de-b816-9e13bb5443ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:20.180635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4dfa9a15-5bb5-44de-b816-9e13bb5443ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:20.181046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b07063a2-e853-482b-a470-ebc7000ffdae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:20.181563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b07063a2-e853-482b-a470-ebc7000ffdae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:20.182278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b07063a2-e853-482b-a470-ebc7000ffdae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:20.182711 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b07063a2-e853-482b-a470-ebc7000ffdae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:20.265290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4dfa9a15-5bb5-44de-b816-9e13bb5443ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:20.274383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4dfa9a15-5bb5-44de-b816-9e13bb5443ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.109542 May 25 16:42:20.275063 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1009/2038] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:20 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:20.336478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b07063a2-e853-482b-a470-ebc7000ffdae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:20.345031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b07063a2-e853-482b-a470-ebc7000ffdae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.174352 May 25 16:42:20.345616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1030/2039] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:20 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:21.184273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dcd73b99-d1a3-4c15-8ee3-f73c81333eac tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:21.184974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dcd73b99-d1a3-4c15-8ee3-f73c81333eac tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:21.188936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:21.189711 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:21.193363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcd73b99-d1a3-4c15-8ee3-f73c81333eac tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:21.193933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcd73b99-d1a3-4c15-8ee3-f73c81333eac tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:21.194900 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcd73b99-d1a3-4c15-8ee3-f73c81333eac tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:21.195454 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcd73b99-d1a3-4c15-8ee3-f73c81333eac tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:21.197835 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:21.198564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:21.199404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:21.199955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:21.391739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dcd73b99-d1a3-4c15-8ee3-f73c81333eac tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:21.395915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:21.399987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dcd73b99-d1a3-4c15-8ee3-f73c81333eac tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.221994 May 25 16:42:21.400548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1010/2040] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:21 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:21.404814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:21.404992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-43fdb795-1400-4235-a1fc-dce332e3b344 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:21.405577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-43fdb795-1400-4235-a1fc-dce332e3b344 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:21.406345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:21.407546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-db136ce0-ed97-430f-8578-e09d1f0526bc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.223448 May 25 16:42:21.408278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1031/2041] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:21 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:21.411471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43fdb795-1400-4235-a1fc-dce332e3b344 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:21.411924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43fdb795-1400-4235-a1fc-dce332e3b344 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:21.412684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43fdb795-1400-4235-a1fc-dce332e3b344 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:21.413119 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43fdb795-1400-4235-a1fc-dce332e3b344 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:21.419895 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9016594c-6df0-4aa6-8f49-5304c739e8bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:21.420546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9016594c-6df0-4aa6-8f49-5304c739e8bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:21.426545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9016594c-6df0-4aa6-8f49-5304c739e8bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:21.426981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9016594c-6df0-4aa6-8f49-5304c739e8bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:21.427762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9016594c-6df0-4aa6-8f49-5304c739e8bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:21.428187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9016594c-6df0-4aa6-8f49-5304c739e8bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:21.482722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-43fdb795-1400-4235-a1fc-dce332e3b344 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:21.485004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:21.492316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-43fdb795-1400-4235-a1fc-dce332e3b344 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090765 May 25 16:42:21.492924 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1011/2042] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:21 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:21.580178 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9016594c-6df0-4aa6-8f49-5304c739e8bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:21.588768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9016594c-6df0-4aa6-8f49-5304c739e8bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172394 May 25 16:42:21.589346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1032/2043] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:21 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:22.420132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c3cc00f9-b578-4310-a4fe-4d5531cf5958 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:22.420871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c3cc00f9-b578-4310-a4fe-4d5531cf5958 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:22.427199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:22.427972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:22.429688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3cc00f9-b578-4310-a4fe-4d5531cf5958 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:22.430248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3cc00f9-b578-4310-a4fe-4d5531cf5958 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:22.431524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3cc00f9-b578-4310-a4fe-4d5531cf5958 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:22.432291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c3cc00f9-b578-4310-a4fe-4d5531cf5958 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:22.436410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:22.436976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:22.437925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:22.438474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:22.629296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:22.631847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c3cc00f9-b578-4310-a4fe-4d5531cf5958 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:22.638995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:22.639889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c3cc00f9-b578-4310-a4fe-4d5531cf5958 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.225655 May 25 16:42:22.640503 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1012/2044] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:22 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:22.640666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:22.641694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-227cd5d4-09c6-4f1a-8bc5-1ac58ae7620d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.219408 May 25 16:42:22.642450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1033/2045] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:22 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:22.645986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-efbe9839-3f9f-4efc-9149-ea0f49142d9c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:22.646672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-67f6eba2-b72e-4655-8638-99c970f05a55 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:22.647216 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-efbe9839-3f9f-4efc-9149-ea0f49142d9c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:22.647364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-67f6eba2-b72e-4655-8638-99c970f05a55 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:22.653373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-67f6eba2-b72e-4655-8638-99c970f05a55 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:22.653825 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-67f6eba2-b72e-4655-8638-99c970f05a55 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:22.654213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-efbe9839-3f9f-4efc-9149-ea0f49142d9c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:22.654588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-67f6eba2-b72e-4655-8638-99c970f05a55 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:22.654699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-efbe9839-3f9f-4efc-9149-ea0f49142d9c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:22.655021 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-67f6eba2-b72e-4655-8638-99c970f05a55 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:22.655416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-efbe9839-3f9f-4efc-9149-ea0f49142d9c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:22.655841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-efbe9839-3f9f-4efc-9149-ea0f49142d9c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:22.743139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-efbe9839-3f9f-4efc-9149-ea0f49142d9c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:22.751752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-efbe9839-3f9f-4efc-9149-ea0f49142d9c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.110272 May 25 16:42:22.752463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1013/2046] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:22 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:22.809105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-67f6eba2-b72e-4655-8638-99c970f05a55 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:22.822006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-67f6eba2-b72e-4655-8638-99c970f05a55 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.178575 May 25 16:42:22.822792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1034/2047] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:22 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:23.661335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3321681f-3c40-41ce-bbbd-bda4d329a172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:23.662265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3321681f-3c40-41ce-bbbd-bda4d329a172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:23.664616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:23.666632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:23.670852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3321681f-3c40-41ce-bbbd-bda4d329a172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:23.671323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3321681f-3c40-41ce-bbbd-bda4d329a172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:23.672059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3321681f-3c40-41ce-bbbd-bda4d329a172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:23.672482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3321681f-3c40-41ce-bbbd-bda4d329a172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:23.673312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:23.673778 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:23.674555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:23.674992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:23.871765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3321681f-3c40-41ce-bbbd-bda4d329a172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:23.876715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:23.883138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3321681f-3c40-41ce-bbbd-bda4d329a172 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.227875 May 25 16:42:23.883915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1014/2048] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:23 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:23.887569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:23.888627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-238a38f0-1585-4d5a-ae1c-506792f3515a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:23.889230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-238a38f0-1585-4d5a-ae1c-506792f3515a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:23.889404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:23.890681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8309ba6b-9b40-432d-bbeb-2543b1428bbb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.232840 May 25 16:42:23.891340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1035/2049] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:23 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:23.895185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-238a38f0-1585-4d5a-ae1c-506792f3515a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:23.895712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-238a38f0-1585-4d5a-ae1c-506792f3515a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:23.896479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-238a38f0-1585-4d5a-ae1c-506792f3515a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:23.896910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-238a38f0-1585-4d5a-ae1c-506792f3515a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:23.897650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-625008ea-01a7-4df6-ba4c-e3175155cd7f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:23.898267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-625008ea-01a7-4df6-ba4c-e3175155cd7f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:23.904349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-625008ea-01a7-4df6-ba4c-e3175155cd7f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:23.904799 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-625008ea-01a7-4df6-ba4c-e3175155cd7f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:23.905560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-625008ea-01a7-4df6-ba4c-e3175155cd7f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:23.905997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-625008ea-01a7-4df6-ba4c-e3175155cd7f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:23.968367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-238a38f0-1585-4d5a-ae1c-506792f3515a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:23.976590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-238a38f0-1585-4d5a-ae1c-506792f3515a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091609 May 25 16:42:23.977163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1015/2050] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:23 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:24.053345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-625008ea-01a7-4df6-ba4c-e3175155cd7f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:24.062993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-625008ea-01a7-4df6-ba4c-e3175155cd7f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.168892 May 25 16:42:24.063633 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1036/2051] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:23 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:24.904884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1605bd87-6978-4788-815d-55ce94e9524b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:24.905930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1605bd87-6978-4788-815d-55ce94e9524b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:24.911594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:24.912581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:24.917346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1605bd87-6978-4788-815d-55ce94e9524b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:24.918090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1605bd87-6978-4788-815d-55ce94e9524b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:24.919444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1605bd87-6978-4788-815d-55ce94e9524b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:24.920175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1605bd87-6978-4788-815d-55ce94e9524b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:24.923706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:24.924450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:24.925783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:24.926492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:25.131176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1605bd87-6978-4788-815d-55ce94e9524b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:25.135337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:25.140158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1605bd87-6978-4788-815d-55ce94e9524b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.241827 May 25 16:42:25.140962 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1016/2052] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:24 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:25.145451 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-819b34f0-b39a-4fed-8d91-f1af2f6c14e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:25.146120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-819b34f0-b39a-4fed-8d91-f1af2f6c14e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:25.151018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:25.152478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-819b34f0-b39a-4fed-8d91-f1af2f6c14e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:25.152776 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:25.152886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-819b34f0-b39a-4fed-8d91-f1af2f6c14e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:25.153690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-819b34f0-b39a-4fed-8d91-f1af2f6c14e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:25.153924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-15f5ee86-c8ec-4bd8-9275-a1f7eedd01c0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.248809 May 25 16:42:25.154136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-819b34f0-b39a-4fed-8d91-f1af2f6c14e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:25.154878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1037/2053] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:24 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:25.159743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cb0e0cf7-6305-44fd-b9d5-be28791f2cee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:25.160552 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cb0e0cf7-6305-44fd-b9d5-be28791f2cee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:25.168051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb0e0cf7-6305-44fd-b9d5-be28791f2cee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:25.168470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb0e0cf7-6305-44fd-b9d5-be28791f2cee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:25.169223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb0e0cf7-6305-44fd-b9d5-be28791f2cee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:25.169952 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cb0e0cf7-6305-44fd-b9d5-be28791f2cee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:25.185903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:25.244043 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-819b34f0-b39a-4fed-8d91-f1af2f6c14e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:25.252075 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-819b34f0-b39a-4fed-8d91-f1af2f6c14e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.110064 May 25 16:42:25.252642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1017/2054] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:25 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:25.315007 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cb0e0cf7-6305-44fd-b9d5-be28791f2cee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:25.323642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cb0e0cf7-6305-44fd-b9d5-be28791f2cee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.167703 May 25 16:42:25.324223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1038/2055] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:25 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:26.160573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4ec6cac5-ab2e-46b0-8c63-f2e2b7af36e6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:26.161456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4ec6cac5-ab2e-46b0-8c63-f2e2b7af36e6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:26.170147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4ec6cac5-ab2e-46b0-8c63-f2e2b7af36e6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:26.170598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4ec6cac5-ab2e-46b0-8c63-f2e2b7af36e6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:26.171405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4ec6cac5-ab2e-46b0-8c63-f2e2b7af36e6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:26.171843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4ec6cac5-ab2e-46b0-8c63-f2e2b7af36e6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:26.173199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:26.173810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:26.181094 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:26.181553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:26.182291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:26.182725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:26.364042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4ec6cac5-ab2e-46b0-8c63-f2e2b7af36e6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:26.367212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:26.372578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4ec6cac5-ab2e-46b0-8c63-f2e2b7af36e6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.217901 May 25 16:42:26.373339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1018/2056] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:26 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:26.376164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:26.377703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:26.377841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1c93d326-c7a0-449f-9fe8-5e0c5eaa79e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:26.378380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1c93d326-c7a0-449f-9fe8-5e0c5eaa79e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:26.378865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0cf8d303-2ea9-4137-aeaf-491736555eec tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.209581 May 25 16:42:26.383205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1039/2057] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:26.383690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-13c039a2-b665-4291-ad8b-e5efb6f96359 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:26.384289 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1c93d326-c7a0-449f-9fe8-5e0c5eaa79e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:26.384426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-13c039a2-b665-4291-ad8b-e5efb6f96359 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:26.384969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1c93d326-c7a0-449f-9fe8-5e0c5eaa79e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:26.385754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1c93d326-c7a0-449f-9fe8-5e0c5eaa79e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:26.386146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1c93d326-c7a0-449f-9fe8-5e0c5eaa79e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:26.390166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13c039a2-b665-4291-ad8b-e5efb6f96359 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:26.390613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13c039a2-b665-4291-ad8b-e5efb6f96359 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:26.391404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13c039a2-b665-4291-ad8b-e5efb6f96359 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:26.391842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13c039a2-b665-4291-ad8b-e5efb6f96359 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:26.475052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1c93d326-c7a0-449f-9fe8-5e0c5eaa79e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:26.483372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1c93d326-c7a0-449f-9fe8-5e0c5eaa79e9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.109022 May 25 16:42:26.484240 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1019/2058] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:26.545391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-13c039a2-b665-4291-ad8b-e5efb6f96359 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:26.553637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-13c039a2-b665-4291-ad8b-e5efb6f96359 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173239 May 25 16:42:26.554209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1040/2059] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:26 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:27.391388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a0f01eab-ceef-48c6-8f72-e88a4c415ce2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:27.392242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a0f01eab-ceef-48c6-8f72-e88a4c415ce2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:27.398145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:27.398858 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:27.399902 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0f01eab-ceef-48c6-8f72-e88a4c415ce2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:27.400405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0f01eab-ceef-48c6-8f72-e88a4c415ce2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:27.401327 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0f01eab-ceef-48c6-8f72-e88a4c415ce2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:27.401815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a0f01eab-ceef-48c6-8f72-e88a4c415ce2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:27.406403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:27.406869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:27.407802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:27.408351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:27.606837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a0f01eab-ceef-48c6-8f72-e88a4c415ce2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:27.611361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:27.615349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a0f01eab-ceef-48c6-8f72-e88a4c415ce2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.228538 May 25 16:42:27.615910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1020/2060] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:27 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:27.620132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:27.620310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b5863d58-8fc5-4ddf-b878-bdc353af2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:27.620865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b5863d58-8fc5-4ddf-b878-bdc353af2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:27.621655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:27.622827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-34e60cec-2741-40c5-ab71-d969cc61c7b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.229228 May 25 16:42:27.623821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1041/2061] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:42:27.628508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5863d58-8fc5-4ddf-b878-bdc353af2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:27.629168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a460ea54-fb16-4591-90ad-4152ed97609e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:27.629314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5863d58-8fc5-4ddf-b878-bdc353af2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:27.629957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a460ea54-fb16-4591-90ad-4152ed97609e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:27.630196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5863d58-8fc5-4ddf-b878-bdc353af2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:27.630623 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b5863d58-8fc5-4ddf-b878-bdc353af2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:27.639691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a460ea54-fb16-4591-90ad-4152ed97609e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:27.640152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a460ea54-fb16-4591-90ad-4152ed97609e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:27.640972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a460ea54-fb16-4591-90ad-4152ed97609e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:27.641523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a460ea54-fb16-4591-90ad-4152ed97609e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:27.718119 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b5863d58-8fc5-4ddf-b878-bdc353af2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:27.726411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b5863d58-8fc5-4ddf-b878-bdc353af2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.109519 May 25 16:42:27.726980 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1021/2062] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:27 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:27.794647 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a460ea54-fb16-4591-90ad-4152ed97609e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:27.808161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a460ea54-fb16-4591-90ad-4152ed97609e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.183391 May 25 16:42:27.808753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1042/2063] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:27 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:28.633296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-34b39195-ed16-4909-bc22-4c25ea711e92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:28.633871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-34b39195-ed16-4909-bc22-4c25ea711e92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:28.640384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34b39195-ed16-4909-bc22-4c25ea711e92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:28.640836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34b39195-ed16-4909-bc22-4c25ea711e92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:28.641604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34b39195-ed16-4909-bc22-4c25ea711e92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:28.642026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-34b39195-ed16-4909-bc22-4c25ea711e92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:28.643323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:28.643939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:28.650715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:28.651221 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:28.652081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:28.652524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:28.834046 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-34b39195-ed16-4909-bc22-4c25ea711e92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:28.841488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:28.843816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-34b39195-ed16-4909-bc22-4c25ea711e92 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.214491 May 25 16:42:28.844599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1022/2064] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:28 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:28.849338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1dd3fa75-ecef-45d0-be7f-09b3d83daa00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:28.850139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1dd3fa75-ecef-45d0-be7f-09b3d83daa00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:28.855090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:28.856292 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1dd3fa75-ecef-45d0-be7f-09b3d83daa00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:28.856601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:28.856731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1dd3fa75-ecef-45d0-be7f-09b3d83daa00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:28.857529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1dd3fa75-ecef-45d0-be7f-09b3d83daa00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:28.857794 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ce60d881-c846-4e9f-8f69-83acf1fbaf39 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1802 microversion: 2.20 time: 0.218481 May 25 16:42:28.857948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1dd3fa75-ecef-45d0-be7f-09b3d83daa00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:28.858348 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1043/2065] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1802 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:28.862741 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2027d1b8-db57-4554-a95d-9f62fcdcbd0d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:28.863511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2027d1b8-db57-4554-a95d-9f62fcdcbd0d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:28.869640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2027d1b8-db57-4554-a95d-9f62fcdcbd0d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:28.870087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2027d1b8-db57-4554-a95d-9f62fcdcbd0d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:28.870834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2027d1b8-db57-4554-a95d-9f62fcdcbd0d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:28.871282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2027d1b8-db57-4554-a95d-9f62fcdcbd0d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:28.935893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1dd3fa75-ecef-45d0-be7f-09b3d83daa00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:28.944172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1dd3fa75-ecef-45d0-be7f-09b3d83daa00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.098474 May 25 16:42:28.944851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1023/2066] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:29.018581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2027d1b8-db57-4554-a95d-9f62fcdcbd0d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:29.029983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2027d1b8-db57-4554-a95d-9f62fcdcbd0d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.170563 May 25 16:42:29.030558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1044/2067] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:28 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:29.862912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aa6e3c7e-6799-4497-a074-a6a842d7bcab tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:29.865929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aa6e3c7e-6799-4497-a074-a6a842d7bcab tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:29.876402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa6e3c7e-6799-4497-a074-a6a842d7bcab tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:29.878314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:29.878932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:29.879448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa6e3c7e-6799-4497-a074-a6a842d7bcab tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:29.880551 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa6e3c7e-6799-4497-a074-a6a842d7bcab tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:29.880948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa6e3c7e-6799-4497-a074-a6a842d7bcab tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:29.885908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:29.886359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:29.887085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:29.887749 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:30.003385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:30.012350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:30.013843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:30.014999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a5210d5d-6146-4782-9608-4f5946ee545d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1734 microversion: 2.20 time: 0.140491 May 25 16:42:30.015586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1045/2068] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:29 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1734 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:30.019913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c5c02e22-2762-42e3-b8a5-b12d2197ecac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:30.020510 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c5c02e22-2762-42e3-b8a5-b12d2197ecac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:30.026511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5c02e22-2762-42e3-b8a5-b12d2197ecac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:30.026981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5c02e22-2762-42e3-b8a5-b12d2197ecac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:30.027765 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5c02e22-2762-42e3-b8a5-b12d2197ecac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:30.028194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c5c02e22-2762-42e3-b8a5-b12d2197ecac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:30.107766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c5c02e22-2762-42e3-b8a5-b12d2197ecac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:30.114529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa6e3c7e-6799-4497-a074-a6a842d7bcab tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:30.117089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c5c02e22-2762-42e3-b8a5-b12d2197ecac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.100524 May 25 16:42:30.118586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1046/2069] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:30 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:30.124233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aa6e3c7e-6799-4497-a074-a6a842d7bcab tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.265603 May 25 16:42:30.125840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1024/2070] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:29 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:30.290067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-42178377-69c0-492b-a676-239a54838d23 req-63c4e299-c336-452d-8370-dbff79a0f80c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4", "name": "network-vif-deleted", "tag": "4d5ed500-2623-48b8-bc81-a72eb4ffeb2d"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:42:30.296397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a80bfbbd-b548-4667-bc18-23ab956c61bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:30.297165 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a80bfbbd-b548-4667-bc18-23ab956c61bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:30.309960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a80bfbbd-b548-4667-bc18-23ab956c61bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:30.312000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-42178377-69c0-492b-a676-239a54838d23 req-63c4e299-c336-452d-8370-dbff79a0f80c service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:30.313113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a80bfbbd-b548-4667-bc18-23ab956c61bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:30.313913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a80bfbbd-b548-4667-bc18-23ab956c61bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:30.314343 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a80bfbbd-b548-4667-bc18-23ab956c61bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:30.318869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-42178377-69c0-492b-a676-239a54838d23 req-63c4e299-c336-452d-8370-dbff79a0f80c service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:30.341553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-42178377-69c0-492b-a676-239a54838d23 req-63c4e299-c336-452d-8370-dbff79a0f80c service nova] Creating event network-vif-deleted:4d5ed500-2623-48b8-bc81-a72eb4ffeb2d for instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 on hv-cinder-84327 May 25 16:42:30.347039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-42178377-69c0-492b-a676-239a54838d23 req-63c4e299-c336-452d-8370-dbff79a0f80c service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069385 May 25 16:42:30.348391 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1025/2071] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:42:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:42:30.482593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a80bfbbd-b548-4667-bc18-23ab956c61bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:30.490813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a80bfbbd-b548-4667-bc18-23ab956c61bd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.372180 May 25 16:42:30.491398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1047/2072] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:30 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:31.034223 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:31.035141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:31.042487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:31.043009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:31.043843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:31.044230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:31.123684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:31.134022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dabc525b-8869-42aa-a733-601281d85c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:31.134722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dabc525b-8869-42aa-a733-601281d85c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:31.135603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:31.137491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:31.138832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dcb0e342-5bcc-48da-8e71-fbc75f7f56b6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.108930 May 25 16:42:31.139407 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1026/2073] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:31 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:31.141657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dabc525b-8869-42aa-a733-601281d85c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:31.142117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dabc525b-8869-42aa-a733-601281d85c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:31.142889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dabc525b-8869-42aa-a733-601281d85c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:31.143356 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dabc525b-8869-42aa-a733-601281d85c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:31.143797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dcc0e121-6ad8-4fed-a164-1d984487e0f4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:31.144393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dcc0e121-6ad8-4fed-a164-1d984487e0f4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:31.152183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcc0e121-6ad8-4fed-a164-1d984487e0f4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:31.152625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcc0e121-6ad8-4fed-a164-1d984487e0f4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:31.153379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcc0e121-6ad8-4fed-a164-1d984487e0f4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:31.153804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dcc0e121-6ad8-4fed-a164-1d984487e0f4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:31.228514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dabc525b-8869-42aa-a733-601281d85c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:31.237212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dabc525b-8869-42aa-a733-601281d85c54 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.107128 May 25 16:42:31.238126 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1048/2074] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:31 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:42:31.299070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dcc0e121-6ad8-4fed-a164-1d984487e0f4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:31.307736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dcc0e121-6ad8-4fed-a164-1d984487e0f4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.167289 May 25 16:42:31.308316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1027/2075] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:31 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:31.508913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8659b6a2-4ecb-413c-af70-895dd423a2c3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:31.509691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8659b6a2-4ecb-413c-af70-895dd423a2c3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:31.516576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8659b6a2-4ecb-413c-af70-895dd423a2c3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:31.517020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8659b6a2-4ecb-413c-af70-895dd423a2c3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:31.517786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8659b6a2-4ecb-413c-af70-895dd423a2c3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:31.518216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8659b6a2-4ecb-413c-af70-895dd423a2c3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:31.653302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8659b6a2-4ecb-413c-af70-895dd423a2c3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:31.661974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8659b6a2-4ecb-413c-af70-895dd423a2c3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.157107 May 25 16:42:31.662524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1049/2076] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:31 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:32.157111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:32.157727 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:32.165360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:32.165840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:32.166577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:32.167015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:32.239904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:32.249108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:32.250700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:32.251885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0ecfdaa6-4fb1-43e7-b49e-535d217063bb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098770 May 25 16:42:32.252436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1028/2077] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:32 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:32.255092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1ea7d5bd-734d-4fcb-b05e-b0831aa56e2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:32.255782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1ea7d5bd-734d-4fcb-b05e-b0831aa56e2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:32.262498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ea7d5bd-734d-4fcb-b05e-b0831aa56e2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:32.262934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ea7d5bd-734d-4fcb-b05e-b0831aa56e2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:32.264076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ea7d5bd-734d-4fcb-b05e-b0831aa56e2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:32.264509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1ea7d5bd-734d-4fcb-b05e-b0831aa56e2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:32.324856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0e9eba39-c5d0-4699-af60-ced4287e163f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:32.325352 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0e9eba39-c5d0-4699-af60-ced4287e163f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:32.328851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1ea7d5bd-734d-4fcb-b05e-b0831aa56e2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:32.332439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e9eba39-c5d0-4699-af60-ced4287e163f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:32.333071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e9eba39-c5d0-4699-af60-ced4287e163f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:32.334866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e9eba39-c5d0-4699-af60-ced4287e163f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:32.335540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0e9eba39-c5d0-4699-af60-ced4287e163f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:32.338729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:32.339057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1ea7d5bd-734d-4fcb-b05e-b0831aa56e2b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088484 May 25 16:42:32.340169 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1050/2078] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:32 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:42:32.476638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0e9eba39-c5d0-4699-af60-ced4287e163f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:32.487420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0e9eba39-c5d0-4699-af60-ced4287e163f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.165946 May 25 16:42:32.487849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1029/2079] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:32 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:42:32.679784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6415616b-f69b-4f6b-ab1c-1043c80537f2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:32.680397 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6415616b-f69b-4f6b-ab1c-1043c80537f2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:32.686869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6415616b-f69b-4f6b-ab1c-1043c80537f2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:32.687343 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6415616b-f69b-4f6b-ab1c-1043c80537f2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:32.688101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6415616b-f69b-4f6b-ab1c-1043c80537f2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:32.688567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6415616b-f69b-4f6b-ab1c-1043c80537f2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:32.830677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6415616b-f69b-4f6b-ab1c-1043c80537f2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:32.843263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6415616b-f69b-4f6b-ab1c-1043c80537f2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.167018 May 25 16:42:32.844048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1051/2080] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:32 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:33.271957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:33.272763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:33.279964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:33.280417 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:33.281168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:33.281599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:33.351468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:33.355892 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-97028124-bf87-464f-a093-187fe1c88755 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:33.356509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-97028124-bf87-464f-a093-187fe1c88755 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:33.360587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:33.362091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:33.363291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-821c6ce8-1548-4682-b27b-7bb6646713ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.096519 May 25 16:42:33.363419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97028124-bf87-464f-a093-187fe1c88755 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:33.363803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97028124-bf87-464f-a093-187fe1c88755 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:33.363950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1030/2081] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:33 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:33.364622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97028124-bf87-464f-a093-187fe1c88755 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:33.365438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97028124-bf87-464f-a093-187fe1c88755 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:33.430980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-97028124-bf87-464f-a093-187fe1c88755 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:33.439565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-97028124-bf87-464f-a093-187fe1c88755 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.087456 May 25 16:42:33.440136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1052/2082] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:33 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:33.505027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f95c02d5-18d8-41c4-9493-3f4529f609ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:33.505605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f95c02d5-18d8-41c4-9493-3f4529f609ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:33.512335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f95c02d5-18d8-41c4-9493-3f4529f609ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:33.512782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f95c02d5-18d8-41c4-9493-3f4529f609ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:33.513540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f95c02d5-18d8-41c4-9493-3f4529f609ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:33.513943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f95c02d5-18d8-41c4-9493-3f4529f609ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:33.658131 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f95c02d5-18d8-41c4-9493-3f4529f609ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:33.666782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f95c02d5-18d8-41c4-9493-3f4529f609ad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.165605 May 25 16:42:33.667358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1031/2083] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:33 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:33.868875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-df819666-ee85-435b-8461-7c97576a3c47 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:33.869404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-df819666-ee85-435b-8461-7c97576a3c47 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:33.875963 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df819666-ee85-435b-8461-7c97576a3c47 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:33.876531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df819666-ee85-435b-8461-7c97576a3c47 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:33.879972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df819666-ee85-435b-8461-7c97576a3c47 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:33.880440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df819666-ee85-435b-8461-7c97576a3c47 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:34.010872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-df819666-ee85-435b-8461-7c97576a3c47 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:34.019184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-df819666-ee85-435b-8461-7c97576a3c47 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.156731 May 25 16:42:34.019745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1053/2084] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:33 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:34.385060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:34.385578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:34.392476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:34.392923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:34.393681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:34.394110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:34.456957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b86d36f7-7c71-407a-83d7-26d9a1061a43 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:34.457568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b86d36f7-7c71-407a-83d7-26d9a1061a43 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:34.463185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:34.464411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b86d36f7-7c71-407a-83d7-26d9a1061a43 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:34.464867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b86d36f7-7c71-407a-83d7-26d9a1061a43 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:34.465638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b86d36f7-7c71-407a-83d7-26d9a1061a43 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:34.466078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b86d36f7-7c71-407a-83d7-26d9a1061a43 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:34.471938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:34.473452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:34.474599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a8b87b54-996d-4a04-a8b8-afc988cf3b2b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.095814 May 25 16:42:34.475320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1032/2085] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:34 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:34.531055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b86d36f7-7c71-407a-83d7-26d9a1061a43 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:34.539676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b86d36f7-7c71-407a-83d7-26d9a1061a43 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.086360 May 25 16:42:34.540279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1054/2086] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:34 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:42:34.687128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7c67ff3a-5948-4c4a-ac5a-baa69854ba64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:34.687877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7c67ff3a-5948-4c4a-ac5a-baa69854ba64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:34.696200 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c67ff3a-5948-4c4a-ac5a-baa69854ba64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:34.696833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c67ff3a-5948-4c4a-ac5a-baa69854ba64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:34.697873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c67ff3a-5948-4c4a-ac5a-baa69854ba64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:34.698448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c67ff3a-5948-4c4a-ac5a-baa69854ba64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:34.841912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7c67ff3a-5948-4c4a-ac5a-baa69854ba64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:34.853841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7c67ff3a-5948-4c4a-ac5a-baa69854ba64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.171660 May 25 16:42:34.854416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1033/2087] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:34 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:35.040920 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4c6779e3-6bbe-46b2-bb25-f2a19b801cb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:35.041601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4c6779e3-6bbe-46b2-bb25-f2a19b801cb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:35.048635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c6779e3-6bbe-46b2-bb25-f2a19b801cb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:35.049132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c6779e3-6bbe-46b2-bb25-f2a19b801cb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:35.049971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c6779e3-6bbe-46b2-bb25-f2a19b801cb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:35.050452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c6779e3-6bbe-46b2-bb25-f2a19b801cb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:35.053267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:35.200754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4c6779e3-6bbe-46b2-bb25-f2a19b801cb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:35.208815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4c6779e3-6bbe-46b2-bb25-f2a19b801cb9 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.174707 May 25 16:42:35.209572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1055/2088] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:35 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:35.495886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:35.496463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:35.503284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:35.503728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:35.504482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:35.504946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:35.559039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-439a3185-d736-44a9-b801-1962c5b26da2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:35.559685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-439a3185-d736-44a9-b801-1962c5b26da2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:35.565990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-439a3185-d736-44a9-b801-1962c5b26da2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:35.566503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-439a3185-d736-44a9-b801-1962c5b26da2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:35.567281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-439a3185-d736-44a9-b801-1962c5b26da2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:35.567723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-439a3185-d736-44a9-b801-1962c5b26da2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:35.571614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:35.580325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:35.581828 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:35.582966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3534a4a3-d84c-4c74-8c71-0447d5560284 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.092600 May 25 16:42:35.583553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1034/2089] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:35 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:35.629168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-439a3185-d736-44a9-b801-1962c5b26da2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:35.637472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-439a3185-d736-44a9-b801-1962c5b26da2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084938 May 25 16:42:35.638044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1056/2090] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:35 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:42:35.876836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fbf6be72-231c-4e05-bf79-6287fb98a65d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:35.877469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fbf6be72-231c-4e05-bf79-6287fb98a65d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:35.885207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fbf6be72-231c-4e05-bf79-6287fb98a65d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:35.885658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fbf6be72-231c-4e05-bf79-6287fb98a65d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:35.886406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fbf6be72-231c-4e05-bf79-6287fb98a65d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:35.886833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fbf6be72-231c-4e05-bf79-6287fb98a65d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:36.017359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fbf6be72-231c-4e05-bf79-6287fb98a65d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:36.027797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fbf6be72-231c-4e05-bf79-6287fb98a65d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.155205 May 25 16:42:36.028367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1035/2091] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:35 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:36.233719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-93dab6c3-0370-4f70-8432-5042d4f330ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:36.234374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-93dab6c3-0370-4f70-8432-5042d4f330ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:36.242369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-93dab6c3-0370-4f70-8432-5042d4f330ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:36.242922 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-93dab6c3-0370-4f70-8432-5042d4f330ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:36.244095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-93dab6c3-0370-4f70-8432-5042d4f330ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:36.244548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-93dab6c3-0370-4f70-8432-5042d4f330ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:36.393533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-93dab6c3-0370-4f70-8432-5042d4f330ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:36.402503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-93dab6c3-0370-4f70-8432-5042d4f330ee tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173770 May 25 16:42:36.403081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1057/2092] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:36 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:36.606646 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:36.607487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:36.618308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:36.619006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:36.620257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:36.620932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:36.654369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f355f2cf-eed1-47c5-b9af-01af55f55bbb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:36.655042 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f355f2cf-eed1-47c5-b9af-01af55f55bbb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:36.662084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f355f2cf-eed1-47c5-b9af-01af55f55bbb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:36.662502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f355f2cf-eed1-47c5-b9af-01af55f55bbb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:36.663365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f355f2cf-eed1-47c5-b9af-01af55f55bbb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:36.663795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f355f2cf-eed1-47c5-b9af-01af55f55bbb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:36.689965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:36.698747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:36.700276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:36.701393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aa31ee52-b99b-4f6d-a392-8de165f08a20 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.102606 May 25 16:42:36.701954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1036/2093] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:36 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:36.725487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f355f2cf-eed1-47c5-b9af-01af55f55bbb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:36.733702 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f355f2cf-eed1-47c5-b9af-01af55f55bbb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083220 May 25 16:42:36.734271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1058/2094] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:36 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:37.045820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-87ca25eb-9704-467c-b9b4-f41df533cf36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:37.046427 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-87ca25eb-9704-467c-b9b4-f41df533cf36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:37.052861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87ca25eb-9704-467c-b9b4-f41df533cf36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:37.053310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87ca25eb-9704-467c-b9b4-f41df533cf36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:37.054060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87ca25eb-9704-467c-b9b4-f41df533cf36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:37.054490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87ca25eb-9704-467c-b9b4-f41df533cf36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:37.193393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-87ca25eb-9704-467c-b9b4-f41df533cf36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:37.201893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-87ca25eb-9704-467c-b9b4-f41df533cf36 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.160064 May 25 16:42:37.202475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1037/2095] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:37 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:37.429048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-89615b30-5ef1-42d6-9b8a-407070f1578c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:37.429934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-89615b30-5ef1-42d6-9b8a-407070f1578c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:37.440333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89615b30-5ef1-42d6-9b8a-407070f1578c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:37.441084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89615b30-5ef1-42d6-9b8a-407070f1578c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:37.442354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89615b30-5ef1-42d6-9b8a-407070f1578c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:37.443082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-89615b30-5ef1-42d6-9b8a-407070f1578c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:37.576432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-89615b30-5ef1-42d6-9b8a-407070f1578c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:37.584894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-89615b30-5ef1-42d6-9b8a-407070f1578c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162485 May 25 16:42:37.585651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1059/2096] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:37 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:37.723731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:37.724561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:37.735822 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:37.736559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:37.737792 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:37.738500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:37.750231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bbcd6049-52fc-4c5c-b027-f379d4a9b279 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:37.750805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bbcd6049-52fc-4c5c-b027-f379d4a9b279 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:37.757658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bbcd6049-52fc-4c5c-b027-f379d4a9b279 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:37.758101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bbcd6049-52fc-4c5c-b027-f379d4a9b279 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:37.758856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bbcd6049-52fc-4c5c-b027-f379d4a9b279 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:37.759627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bbcd6049-52fc-4c5c-b027-f379d4a9b279 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:37.811072 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:37.820154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:37.821711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:37.822869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-99a6dfb0-9423-4f13-b666-cd8ebc6989a1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.105800 May 25 16:42:37.823426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1038/2097] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:37 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:37.833848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bbcd6049-52fc-4c5c-b027-f379d4a9b279 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:37.842401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bbcd6049-52fc-4c5c-b027-f379d4a9b279 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.095979 May 25 16:42:37.842970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1060/2098] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:37 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:38.223677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0d5ec035-3a88-4160-a26e-051e94cc8602 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:38.224688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0d5ec035-3a88-4160-a26e-051e94cc8602 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:38.232386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d5ec035-3a88-4160-a26e-051e94cc8602 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:38.232840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d5ec035-3a88-4160-a26e-051e94cc8602 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:38.233598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d5ec035-3a88-4160-a26e-051e94cc8602 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:38.234033 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0d5ec035-3a88-4160-a26e-051e94cc8602 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:38.367492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0d5ec035-3a88-4160-a26e-051e94cc8602 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:38.376095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0d5ec035-3a88-4160-a26e-051e94cc8602 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.159088 May 25 16:42:38.376641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1039/2099] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:38 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:38.607283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9ed07794-a1af-4925-ab34-a06efc4aa440 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:38.608273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9ed07794-a1af-4925-ab34-a06efc4aa440 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:38.618641 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ed07794-a1af-4925-ab34-a06efc4aa440 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:38.619042 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ed07794-a1af-4925-ab34-a06efc4aa440 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:38.619853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ed07794-a1af-4925-ab34-a06efc4aa440 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:38.620283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ed07794-a1af-4925-ab34-a06efc4aa440 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:38.772287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9ed07794-a1af-4925-ab34-a06efc4aa440 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:38.784513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9ed07794-a1af-4925-ab34-a06efc4aa440 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.183551 May 25 16:42:38.785303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1061/2100] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:38 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:38.845064 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:38.846118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:38.855528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:38.855954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:38.856752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:38.857182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:38.859575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-707435bc-3f9c-481c-87e8-a17ed8c69479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:38.860154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-707435bc-3f9c-481c-87e8-a17ed8c69479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:38.866350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-707435bc-3f9c-481c-87e8-a17ed8c69479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:38.866821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-707435bc-3f9c-481c-87e8-a17ed8c69479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:38.867625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-707435bc-3f9c-481c-87e8-a17ed8c69479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:38.868057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-707435bc-3f9c-481c-87e8-a17ed8c69479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:38.936034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:38.944859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:38.946388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:38.947591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-175b4ea0-7812-4f43-8cae-6894a7b04095 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.109068 May 25 16:42:38.948157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1040/2101] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:38 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:38.951927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-707435bc-3f9c-481c-87e8-a17ed8c69479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:38.960337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-707435bc-3f9c-481c-87e8-a17ed8c69479 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.104590 May 25 16:42:38.960901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1062/2102] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:38 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:39.396552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-15646102-8869-42da-ba64-c18e5551b4c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:39.397366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-15646102-8869-42da-ba64-c18e5551b4c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:39.407021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-15646102-8869-42da-ba64-c18e5551b4c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:39.407791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-15646102-8869-42da-ba64-c18e5551b4c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:39.408992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-15646102-8869-42da-ba64-c18e5551b4c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:39.409695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-15646102-8869-42da-ba64-c18e5551b4c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:39.542750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-15646102-8869-42da-ba64-c18e5551b4c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:39.551189 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-15646102-8869-42da-ba64-c18e5551b4c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.160661 May 25 16:42:39.551791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1041/2103] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:39 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:39.808326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1bd06e77-304a-4b14-9afd-8a7646c5ba33 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:39.809333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1bd06e77-304a-4b14-9afd-8a7646c5ba33 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:39.820538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1bd06e77-304a-4b14-9afd-8a7646c5ba33 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:39.821302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1bd06e77-304a-4b14-9afd-8a7646c5ba33 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:39.822609 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1bd06e77-304a-4b14-9afd-8a7646c5ba33 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:39.823410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1bd06e77-304a-4b14-9afd-8a7646c5ba33 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:39.965690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:39.966314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:39.968629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1bd06e77-304a-4b14-9afd-8a7646c5ba33 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:39.973125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:39.973583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:39.974398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:39.974831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:39.978535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1bd06e77-304a-4b14-9afd-8a7646c5ba33 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.176856 May 25 16:42:39.979115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1063/2104] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:39 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:39.983566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d6845a26-8bb5-4fb8-a3a3-19635e0b5441 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:39.984161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d6845a26-8bb5-4fb8-a3a3-19635e0b5441 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:39.990486 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6845a26-8bb5-4fb8-a3a3-19635e0b5441 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:39.990906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6845a26-8bb5-4fb8-a3a3-19635e0b5441 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:39.991714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6845a26-8bb5-4fb8-a3a3-19635e0b5441 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:39.992140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d6845a26-8bb5-4fb8-a3a3-19635e0b5441 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:40.045193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:40.054256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:40.055816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:40.056982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-47417416-f5d4-4213-8661-a4d11eafa8fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.095246 May 25 16:42:40.057537 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1042/2105] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:39 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:40.064570 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d6845a26-8bb5-4fb8-a3a3-19635e0b5441 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:40.073262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d6845a26-8bb5-4fb8-a3a3-19635e0b5441 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093164 May 25 16:42:40.073829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1064/2106] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:39 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:40.572821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-81bb3f0f-d418-4ed3-8d75-b00a67fd48c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:40.573672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-81bb3f0f-d418-4ed3-8d75-b00a67fd48c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:40.584022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-81bb3f0f-d418-4ed3-8d75-b00a67fd48c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:40.584754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-81bb3f0f-d418-4ed3-8d75-b00a67fd48c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:40.585982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-81bb3f0f-d418-4ed3-8d75-b00a67fd48c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:40.586634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-81bb3f0f-d418-4ed3-8d75-b00a67fd48c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:40.734509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-81bb3f0f-d418-4ed3-8d75-b00a67fd48c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:40.744749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-81bb3f0f-d418-4ed3-8d75-b00a67fd48c1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.178241 May 25 16:42:40.745021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1043/2107] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:40 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:40.999535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bf9b711f-a9b5-43d3-9096-ec514a972a3b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:41.000444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bf9b711f-a9b5-43d3-9096-ec514a972a3b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:41.010464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bf9b711f-a9b5-43d3-9096-ec514a972a3b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:41.011049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bf9b711f-a9b5-43d3-9096-ec514a972a3b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:41.011850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bf9b711f-a9b5-43d3-9096-ec514a972a3b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:41.012290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bf9b711f-a9b5-43d3-9096-ec514a972a3b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:41.075570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:41.076151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:41.083351 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:41.083808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:41.084631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:41.085097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:41.164297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:41.173420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:41.174925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:41.176084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-eaffe144-b691-4d17-80b7-fb40f440da38 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.104728 May 25 16:42:41.176634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1044/2108] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:41.181368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fdfb530a-07ef-4f71-90de-56f55883efcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:41.181965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fdfb530a-07ef-4f71-90de-56f55883efcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:41.184688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:41.186410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bf9b711f-a9b5-43d3-9096-ec514a972a3b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:41.188283 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fdfb530a-07ef-4f71-90de-56f55883efcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:41.188889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fdfb530a-07ef-4f71-90de-56f55883efcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:41.189647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fdfb530a-07ef-4f71-90de-56f55883efcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:41.190074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fdfb530a-07ef-4f71-90de-56f55883efcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:41.195242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bf9b711f-a9b5-43d3-9096-ec514a972a3b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.202528 May 25 16:42:41.195860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1065/2109] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:40 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:41.253555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fdfb530a-07ef-4f71-90de-56f55883efcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:41.255773 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:41.263434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fdfb530a-07ef-4f71-90de-56f55883efcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085793 May 25 16:42:41.263994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1045/2110] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:41 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:41.768518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-df561500-a069-47db-abd0-27c12ae34d2a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:41.769563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-df561500-a069-47db-abd0-27c12ae34d2a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:41.780145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df561500-a069-47db-abd0-27c12ae34d2a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:41.780880 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df561500-a069-47db-abd0-27c12ae34d2a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:41.782113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df561500-a069-47db-abd0-27c12ae34d2a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:41.782823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df561500-a069-47db-abd0-27c12ae34d2a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:41.922871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-df561500-a069-47db-abd0-27c12ae34d2a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:41.935754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-df561500-a069-47db-abd0-27c12ae34d2a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.173394 May 25 16:42:41.936629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1066/2111] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:41 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:42.194145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:42.194730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:42.201339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:42.201785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:42.202547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:42.202949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:42.215779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-90eb562d-f26f-4a35-8a56-be346b7e1b83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:42.216427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-90eb562d-f26f-4a35-8a56-be346b7e1b83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:42.223384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90eb562d-f26f-4a35-8a56-be346b7e1b83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:42.223839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90eb562d-f26f-4a35-8a56-be346b7e1b83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:42.224603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90eb562d-f26f-4a35-8a56-be346b7e1b83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:42.225046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-90eb562d-f26f-4a35-8a56-be346b7e1b83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:42.276952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:42.285625 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:42.287125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:42.288412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ca41cea8-7c0c-4553-8c52-f4960eb0ccb4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098254 May 25 16:42:42.289333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1046/2112] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:42.293766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a05ad5ea-c046-4db0-a308-a774f5d16290 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:42.294426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a05ad5ea-c046-4db0-a308-a774f5d16290 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:42.300416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a05ad5ea-c046-4db0-a308-a774f5d16290 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:42.300940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a05ad5ea-c046-4db0-a308-a774f5d16290 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:42.301747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a05ad5ea-c046-4db0-a308-a774f5d16290 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:42.302224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a05ad5ea-c046-4db0-a308-a774f5d16290 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:42.372616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a05ad5ea-c046-4db0-a308-a774f5d16290 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:42.380587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a05ad5ea-c046-4db0-a308-a774f5d16290 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090188 May 25 16:42:42.381120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1047/2113] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:42 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:42.390824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-90eb562d-f26f-4a35-8a56-be346b7e1b83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:42.399102 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-90eb562d-f26f-4a35-8a56-be346b7e1b83 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.187442 May 25 16:42:42.399694 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1067/2114] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:42 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:42.962704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9645081c-9cb7-49ca-bd13-841814042e21 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:42.963345 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9645081c-9cb7-49ca-bd13-841814042e21 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:42.970719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9645081c-9cb7-49ca-bd13-841814042e21 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:42.971254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9645081c-9cb7-49ca-bd13-841814042e21 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:42.972097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9645081c-9cb7-49ca-bd13-841814042e21 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:42.972572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9645081c-9cb7-49ca-bd13-841814042e21 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:43.104505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9645081c-9cb7-49ca-bd13-841814042e21 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:43.112779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9645081c-9cb7-49ca-bd13-841814042e21 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.156503 May 25 16:42:43.113348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1048/2115] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:42 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:43.310011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:43.311005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:43.321869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:43.322317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:43.323077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:43.323541 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:43.394677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:43.397895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-532caef3-5b63-4666-822c-a2bd11150f15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:43.398691 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-532caef3-5b63-4666-822c-a2bd11150f15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:43.403528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:43.405025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:43.405290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-532caef3-5b63-4666-822c-a2bd11150f15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:43.405747 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-532caef3-5b63-4666-822c-a2bd11150f15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:43.406154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ba36f78f-efb0-4a32-a781-17ffe77a54fb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.102809 May 25 16:42:43.406546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-532caef3-5b63-4666-822c-a2bd11150f15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:43.415212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1068/2116] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:43.415599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-532caef3-5b63-4666-822c-a2bd11150f15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:43.416582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a1ae9efa-9307-484f-a328-5bdd771d5193 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:43.417198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a1ae9efa-9307-484f-a328-5bdd771d5193 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:43.423513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1ae9efa-9307-484f-a328-5bdd771d5193 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:43.423968 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1ae9efa-9307-484f-a328-5bdd771d5193 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:43.424733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1ae9efa-9307-484f-a328-5bdd771d5193 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:43.425174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1ae9efa-9307-484f-a328-5bdd771d5193 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:43.474073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-532caef3-5b63-4666-822c-a2bd11150f15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:43.482398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-532caef3-5b63-4666-822c-a2bd11150f15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088327 May 25 16:42:43.482945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1049/2117] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:43 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:43.569555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a1ae9efa-9307-484f-a328-5bdd771d5193 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:43.577762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a1ae9efa-9307-484f-a328-5bdd771d5193 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.165268 May 25 16:42:43.578305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1069/2118] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:43 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:44.130609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6fd05193-f348-4928-8293-328a72af9f31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:44.131230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6fd05193-f348-4928-8293-328a72af9f31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:44.138242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6fd05193-f348-4928-8293-328a72af9f31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:44.138693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6fd05193-f348-4928-8293-328a72af9f31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:44.139486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6fd05193-f348-4928-8293-328a72af9f31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:44.139920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6fd05193-f348-4928-8293-328a72af9f31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:44.284661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6fd05193-f348-4928-8293-328a72af9f31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:44.293629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6fd05193-f348-4928-8293-328a72af9f31 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.166975 May 25 16:42:44.294207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1050/2119] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:44 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:44.428533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:44.429546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:44.437039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:44.437488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:44.438249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:44.438684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:44.499462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-97ec3555-1a6f-4532-957a-e03548de3362 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:44.500248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-97ec3555-1a6f-4532-957a-e03548de3362 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:44.503140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:44.507729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97ec3555-1a6f-4532-957a-e03548de3362 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:44.508260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97ec3555-1a6f-4532-957a-e03548de3362 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:44.509128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97ec3555-1a6f-4532-957a-e03548de3362 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:44.509565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97ec3555-1a6f-4532-957a-e03548de3362 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:44.511936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:44.513433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:44.514585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9601ed18-296a-4ed7-aa87-739198907451 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.092732 May 25 16:42:44.515120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1070/2120] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:44 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:44.575917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-97ec3555-1a6f-4532-957a-e03548de3362 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:44.584110 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-97ec3555-1a6f-4532-957a-e03548de3362 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088579 May 25 16:42:44.584684 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1051/2121] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:44 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:44.595877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3cc9e3c2-569a-488e-9286-1abc302befe7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:44.596803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3cc9e3c2-569a-488e-9286-1abc302befe7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:44.606354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cc9e3c2-569a-488e-9286-1abc302befe7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:44.607065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cc9e3c2-569a-488e-9286-1abc302befe7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:44.608224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cc9e3c2-569a-488e-9286-1abc302befe7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:44.608815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3cc9e3c2-569a-488e-9286-1abc302befe7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:44.753903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3cc9e3c2-569a-488e-9286-1abc302befe7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:44.756249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:44.763483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3cc9e3c2-569a-488e-9286-1abc302befe7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172292 May 25 16:42:44.764051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1071/2122] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:44 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:45.314942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f6853b9e-d109-4922-8c7a-a21d63868dce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:45.318795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f6853b9e-d109-4922-8c7a-a21d63868dce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:45.325541 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f6853b9e-d109-4922-8c7a-a21d63868dce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:45.325923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f6853b9e-d109-4922-8c7a-a21d63868dce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:45.326729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f6853b9e-d109-4922-8c7a-a21d63868dce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:45.327171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f6853b9e-d109-4922-8c7a-a21d63868dce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:45.467755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f6853b9e-d109-4922-8c7a-a21d63868dce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:45.476337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f6853b9e-d109-4922-8c7a-a21d63868dce tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.167722 May 25 16:42:45.476903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1052/2123] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:45 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:45.536937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:45.537775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:45.545713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:45.546165 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:45.546927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:45.547410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:45.605155 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-10702324-fbf8-4449-9cfc-26842778ae40 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:45.605699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-10702324-fbf8-4449-9cfc-26842778ae40 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:45.613062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-10702324-fbf8-4449-9cfc-26842778ae40 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:45.613492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-10702324-fbf8-4449-9cfc-26842778ae40 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:45.614266 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-10702324-fbf8-4449-9cfc-26842778ae40 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:45.614749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-10702324-fbf8-4449-9cfc-26842778ae40 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:45.619586 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:45.629020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:45.630816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:45.632234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e35d0973-55b6-4717-a33c-530b2c94f9c8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.102167 May 25 16:42:45.642666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1072/2124] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:45 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:45.681486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-10702324-fbf8-4449-9cfc-26842778ae40 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:45.690259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-10702324-fbf8-4449-9cfc-26842778ae40 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091975 May 25 16:42:45.690835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1053/2125] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:45 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:45.781057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-88b7b64e-a278-4a29-a1fe-f7a294a27ddc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:45.781819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-88b7b64e-a278-4a29-a1fe-f7a294a27ddc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:45.788048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88b7b64e-a278-4a29-a1fe-f7a294a27ddc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:45.788502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88b7b64e-a278-4a29-a1fe-f7a294a27ddc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:45.789576 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88b7b64e-a278-4a29-a1fe-f7a294a27ddc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:45.790009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-88b7b64e-a278-4a29-a1fe-f7a294a27ddc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:45.938705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-88b7b64e-a278-4a29-a1fe-f7a294a27ddc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:45.951405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-88b7b64e-a278-4a29-a1fe-f7a294a27ddc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173992 May 25 16:42:45.952138 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1073/2126] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:45 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:46.497980 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3a6c9556-028a-44c0-b7cc-39c0b31d37b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:46.498984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3a6c9556-028a-44c0-b7cc-39c0b31d37b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:46.509429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a6c9556-028a-44c0-b7cc-39c0b31d37b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:46.509865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a6c9556-028a-44c0-b7cc-39c0b31d37b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:46.510617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a6c9556-028a-44c0-b7cc-39c0b31d37b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:46.511062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a6c9556-028a-44c0-b7cc-39c0b31d37b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:46.642723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3a6c9556-028a-44c0-b7cc-39c0b31d37b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:46.650216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:46.650829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:46.651040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3a6c9556-028a-44c0-b7cc-39c0b31d37b1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.159625 May 25 16:42:46.651643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1054/2127] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:46 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:46.657827 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:46.658278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:46.659043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:46.659515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:46.707433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-255a2863-74d3-4d82-8e3d-15275f809a77 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:46.708064 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-255a2863-74d3-4d82-8e3d-15275f809a77 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:46.714394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-255a2863-74d3-4d82-8e3d-15275f809a77 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:46.714850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-255a2863-74d3-4d82-8e3d-15275f809a77 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:46.715645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-255a2863-74d3-4d82-8e3d-15275f809a77 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:46.716081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-255a2863-74d3-4d82-8e3d-15275f809a77 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:46.725580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:46.734251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:46.735774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:46.736899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9695307b-4d4e-44c6-a5e2-56f8760fe476 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.090669 May 25 16:42:46.737427 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1074/2128] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:46 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:46.776748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-255a2863-74d3-4d82-8e3d-15275f809a77 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:46.784848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-255a2863-74d3-4d82-8e3d-15275f809a77 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.081371 May 25 16:42:46.785404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1055/2129] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:46 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:46.970276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-815ad9b1-1cfc-43cd-b5a0-07e095e872cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:46.970795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-815ad9b1-1cfc-43cd-b5a0-07e095e872cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:46.977707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-815ad9b1-1cfc-43cd-b5a0-07e095e872cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:46.978155 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-815ad9b1-1cfc-43cd-b5a0-07e095e872cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:46.978909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-815ad9b1-1cfc-43cd-b5a0-07e095e872cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:46.979365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-815ad9b1-1cfc-43cd-b5a0-07e095e872cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:47.124052 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-815ad9b1-1cfc-43cd-b5a0-07e095e872cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:47.132599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-815ad9b1-1cfc-43cd-b5a0-07e095e872cc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.166372 May 25 16:42:47.133206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1075/2130] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:46 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:47.672291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4d286617-0d39-4002-bcb5-d4b7f2642cd3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:47.673269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4d286617-0d39-4002-bcb5-d4b7f2642cd3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:47.684371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d286617-0d39-4002-bcb5-d4b7f2642cd3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:47.685116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d286617-0d39-4002-bcb5-d4b7f2642cd3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:47.686386 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d286617-0d39-4002-bcb5-d4b7f2642cd3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:47.687096 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d286617-0d39-4002-bcb5-d4b7f2642cd3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:47.755600 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:47.756229 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:47.758198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:47.763861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:47.764310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:47.765079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:47.765550 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:47.838635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:47.843316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4d286617-0d39-4002-bcb5-d4b7f2642cd3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:47.847476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:47.848955 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:47.850117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9035ddd2-f63b-44e7-af0d-c7dbb378a318 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098780 May 25 16:42:47.850655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1076/2131] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:47.851940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4d286617-0d39-4002-bcb5-d4b7f2642cd3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.186061 May 25 16:42:47.852619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1056/2132] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:47 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:47.855333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e66bcdeb-3543-47cb-aad2-49b43cff2f22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:47.855939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e66bcdeb-3543-47cb-aad2-49b43cff2f22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:47.861963 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e66bcdeb-3543-47cb-aad2-49b43cff2f22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:47.862413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e66bcdeb-3543-47cb-aad2-49b43cff2f22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:47.863249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e66bcdeb-3543-47cb-aad2-49b43cff2f22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:47.863687 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e66bcdeb-3543-47cb-aad2-49b43cff2f22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:47.928658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e66bcdeb-3543-47cb-aad2-49b43cff2f22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:47.937101 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e66bcdeb-3543-47cb-aad2-49b43cff2f22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085483 May 25 16:42:47.937691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1077/2133] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:47 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:48.154464 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fd14e591-2b56-4630-9fe8-2c43f97eb1ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:48.155523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fd14e591-2b56-4630-9fe8-2c43f97eb1ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:48.166251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd14e591-2b56-4630-9fe8-2c43f97eb1ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:48.166983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd14e591-2b56-4630-9fe8-2c43f97eb1ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:48.168264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd14e591-2b56-4630-9fe8-2c43f97eb1ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:48.168979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fd14e591-2b56-4630-9fe8-2c43f97eb1ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:48.307972 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fd14e591-2b56-4630-9fe8-2c43f97eb1ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:48.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fd14e591-2b56-4630-9fe8-2c43f97eb1ae tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172466 May 25 16:42:48.320969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1057/2134] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:48 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:48.871447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:48.872149 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-19d1e9d7-32e7-45a3-a459-f5af6202ae68 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:48.872602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:48.872744 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-19d1e9d7-32e7-45a3-a459-f5af6202ae68 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:48.880328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19d1e9d7-32e7-45a3-a459-f5af6202ae68 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:48.880855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19d1e9d7-32e7-45a3-a459-f5af6202ae68 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:48.881750 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19d1e9d7-32e7-45a3-a459-f5af6202ae68 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:48.882260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-19d1e9d7-32e7-45a3-a459-f5af6202ae68 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:48.882437 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:48.882807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:48.883745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:48.884259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:48.971041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:48.979908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:48.981697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:48.982837 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-806169dd-4866-4ca9-9683-13d638a7fa63 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.116944 May 25 16:42:48.983712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1078/2135] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:48.988104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2176d635-be1b-482f-9d54-73bb34c8a93e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:48.988705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2176d635-be1b-482f-9d54-73bb34c8a93e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:48.994816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2176d635-be1b-482f-9d54-73bb34c8a93e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:48.995299 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2176d635-be1b-482f-9d54-73bb34c8a93e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:48.996059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2176d635-be1b-482f-9d54-73bb34c8a93e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:48.996488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2176d635-be1b-482f-9d54-73bb34c8a93e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:49.068891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2176d635-be1b-482f-9d54-73bb34c8a93e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:49.072269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-19d1e9d7-32e7-45a3-a459-f5af6202ae68 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:49.077750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2176d635-be1b-482f-9d54-73bb34c8a93e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092996 May 25 16:42:49.078320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1079/2136] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:48 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:49.081291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-19d1e9d7-32e7-45a3-a459-f5af6202ae68 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.214832 May 25 16:42:49.081847 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1058/2137] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:48 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:49.345863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-24dfa5ba-73d3-4499-a901-cec15352168d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:49.346865 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-24dfa5ba-73d3-4499-a901-cec15352168d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:49.355565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24dfa5ba-73d3-4499-a901-cec15352168d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:49.356009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24dfa5ba-73d3-4499-a901-cec15352168d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:49.356754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24dfa5ba-73d3-4499-a901-cec15352168d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:49.357185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-24dfa5ba-73d3-4499-a901-cec15352168d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:49.495439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-24dfa5ba-73d3-4499-a901-cec15352168d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:49.503643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-24dfa5ba-73d3-4499-a901-cec15352168d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.164300 May 25 16:42:49.504219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1080/2138] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:49 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:50.004514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:50.005461 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:50.015116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:50.015601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:50.016371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:50.016808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:50.083117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:50.097229 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d2c3ea7b-1193-4f48-a88f-43337d62b5bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:50.097989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:50.098184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d2c3ea7b-1193-4f48-a88f-43337d62b5bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:50.099829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:50.101524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-135d304f-bc5a-413a-9494-34c4b641b567 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.103498 May 25 16:42:50.102484 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1059/2139] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:50.106793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9f1e87a7-d5c3-4c8c-9ecc-f36445272fb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:50.107429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9f1e87a7-d5c3-4c8c-9ecc-f36445272fb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:50.113679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9f1e87a7-d5c3-4c8c-9ecc-f36445272fb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:50.114107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9f1e87a7-d5c3-4c8c-9ecc-f36445272fb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:50.114860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9f1e87a7-d5c3-4c8c-9ecc-f36445272fb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:50.115360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9f1e87a7-d5c3-4c8c-9ecc-f36445272fb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:50.115796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2c3ea7b-1193-4f48-a88f-43337d62b5bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:50.116247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2c3ea7b-1193-4f48-a88f-43337d62b5bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:50.117008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2c3ea7b-1193-4f48-a88f-43337d62b5bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:50.117457 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2c3ea7b-1193-4f48-a88f-43337d62b5bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:50.205350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d2c3ea7b-1193-4f48-a88f-43337d62b5bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:50.213797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d2c3ea7b-1193-4f48-a88f-43337d62b5bc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.122481 May 25 16:42:50.214391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1081/2140] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:50 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:50.267973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9f1e87a7-d5c3-4c8c-9ecc-f36445272fb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:50.279305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9f1e87a7-d5c3-4c8c-9ecc-f36445272fb5 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.175788 May 25 16:42:50.279889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1060/2141] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:50 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:50.525099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-727ca407-e606-425c-ab28-78f32bee94df tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:50.525816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-727ca407-e606-425c-ab28-78f32bee94df tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:50.533313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-727ca407-e606-425c-ab28-78f32bee94df tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:50.533820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-727ca407-e606-425c-ab28-78f32bee94df tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:50.534743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-727ca407-e606-425c-ab28-78f32bee94df tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:50.535290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-727ca407-e606-425c-ab28-78f32bee94df tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:50.683733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-727ca407-e606-425c-ab28-78f32bee94df tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:50.692302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-727ca407-e606-425c-ab28-78f32bee94df tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173732 May 25 16:42:50.692879 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1082/2142] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:50 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:51.121547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:51.122151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:51.129302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:51.129753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:51.130508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:51.130935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:51.195704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:51.204302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:51.205802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:51.206931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-db963a47-eace-4858-851c-d11e32e6400d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.089630 May 25 16:42:51.207533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1061/2143] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:51.233078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-26c1ce0a-b997-4c55-9e58-72a84373f80e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:51.233663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-26c1ce0a-b997-4c55-9e58-72a84373f80e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:51.243455 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26c1ce0a-b997-4c55-9e58-72a84373f80e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:51.243910 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26c1ce0a-b997-4c55-9e58-72a84373f80e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:51.244643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26c1ce0a-b997-4c55-9e58-72a84373f80e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:51.245080 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26c1ce0a-b997-4c55-9e58-72a84373f80e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:51.306270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c7751865-9e88-456c-bf1e-4a2987ac115a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:51.306471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-26c1ce0a-b997-4c55-9e58-72a84373f80e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:51.306900 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c7751865-9e88-456c-bf1e-4a2987ac115a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:51.313516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7751865-9e88-456c-bf1e-4a2987ac115a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:51.313969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7751865-9e88-456c-bf1e-4a2987ac115a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:51.314731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7751865-9e88-456c-bf1e-4a2987ac115a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:51.314879 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-26c1ce0a-b997-4c55-9e58-72a84373f80e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.087736 May 25 16:42:51.315184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7751865-9e88-456c-bf1e-4a2987ac115a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:51.323199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1083/2144] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:51 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:51.330339 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:51.453741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c7751865-9e88-456c-bf1e-4a2987ac115a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:51.465921 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c7751865-9e88-456c-bf1e-4a2987ac115a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.163382 May 25 16:42:51.466477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1062/2145] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:51 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:51.713923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e5ee8acd-253c-41e6-8557-e33a69f26926 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:51.714798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e5ee8acd-253c-41e6-8557-e33a69f26926 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:51.725064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5ee8acd-253c-41e6-8557-e33a69f26926 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:51.725806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5ee8acd-253c-41e6-8557-e33a69f26926 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:51.727064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5ee8acd-253c-41e6-8557-e33a69f26926 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:51.727826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e5ee8acd-253c-41e6-8557-e33a69f26926 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:51.862348 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e5ee8acd-253c-41e6-8557-e33a69f26926 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:51.870725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e5ee8acd-253c-41e6-8557-e33a69f26926 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.163408 May 25 16:42:51.871332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1084/2146] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:51 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:52.228243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:52.228900 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:52.236834 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:52.237329 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:52.238148 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:52.238647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:52.309230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:52.318053 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:52.319595 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:52.320749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dc75f602-29c8-44aa-b86e-d174c1bb03ca tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098951 May 25 16:42:52.321362 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1063/2147] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:52.332713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8b6a6aaa-1902-4de3-a43a-b22fde841afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:52.333410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8b6a6aaa-1902-4de3-a43a-b22fde841afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:52.343130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b6a6aaa-1902-4de3-a43a-b22fde841afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:52.343785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b6a6aaa-1902-4de3-a43a-b22fde841afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:52.344862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b6a6aaa-1902-4de3-a43a-b22fde841afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:52.345464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8b6a6aaa-1902-4de3-a43a-b22fde841afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:52.408001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8b6a6aaa-1902-4de3-a43a-b22fde841afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:52.416701 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8b6a6aaa-1902-4de3-a43a-b22fde841afd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088461 May 25 16:42:52.417317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1085/2148] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:52 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:52.487346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0ea2c6fa-cbcf-4ba0-a163-fd3af01585d0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:52.488360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0ea2c6fa-cbcf-4ba0-a163-fd3af01585d0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:52.498720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ea2c6fa-cbcf-4ba0-a163-fd3af01585d0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:52.499547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ea2c6fa-cbcf-4ba0-a163-fd3af01585d0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:52.500809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ea2c6fa-cbcf-4ba0-a163-fd3af01585d0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:52.501536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ea2c6fa-cbcf-4ba0-a163-fd3af01585d0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:52.646908 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ea2c6fa-cbcf-4ba0-a163-fd3af01585d0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:52.655766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0ea2c6fa-cbcf-4ba0-a163-fd3af01585d0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.174933 May 25 16:42:52.656310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1064/2149] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:52 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:52.897885 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8fd729a0-b3b5-488d-8f04-48b569b03389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:52.898806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8fd729a0-b3b5-488d-8f04-48b569b03389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:52.909013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8fd729a0-b3b5-488d-8f04-48b569b03389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:52.909751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8fd729a0-b3b5-488d-8f04-48b569b03389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:52.911039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8fd729a0-b3b5-488d-8f04-48b569b03389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:52.911806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8fd729a0-b3b5-488d-8f04-48b569b03389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:53.041370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8fd729a0-b3b5-488d-8f04-48b569b03389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:53.053034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8fd729a0-b3b5-488d-8f04-48b569b03389 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.161422 May 25 16:42:53.053849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1086/2150] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:52 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:53.340749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:53.341363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:53.348365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:53.348850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:53.349609 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:53.350042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:53.416343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:53.425242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:53.426717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:53.427969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-da533102-9ad8-45ea-933d-112928b09bd6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.091446 May 25 16:42:53.428527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1065/2151] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:53 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:53.433387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5e3a641f-2866-480b-8868-baa005003e2f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:53.433997 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5e3a641f-2866-480b-8868-baa005003e2f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:53.440891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5e3a641f-2866-480b-8868-baa005003e2f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:53.441339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5e3a641f-2866-480b-8868-baa005003e2f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:53.442095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5e3a641f-2866-480b-8868-baa005003e2f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:53.442558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5e3a641f-2866-480b-8868-baa005003e2f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:53.509329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5e3a641f-2866-480b-8868-baa005003e2f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:53.517483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5e3a641f-2866-480b-8868-baa005003e2f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088172 May 25 16:42:53.518047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1087/2152] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:53 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:53.677705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2a691d79-af7a-449e-b48c-09e1a2d74da6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:53.678635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2a691d79-af7a-449e-b48c-09e1a2d74da6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:53.688901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a691d79-af7a-449e-b48c-09e1a2d74da6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:53.689648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a691d79-af7a-449e-b48c-09e1a2d74da6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:53.690894 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a691d79-af7a-449e-b48c-09e1a2d74da6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:53.691685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2a691d79-af7a-449e-b48c-09e1a2d74da6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:53.835827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2a691d79-af7a-449e-b48c-09e1a2d74da6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:53.844531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2a691d79-af7a-449e-b48c-09e1a2d74da6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.173599 May 25 16:42:53.845105 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1066/2153] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:53 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:54.077244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-655dfd43-dbb8-4059-aba7-3f91434868b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:54.078580 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-655dfd43-dbb8-4059-aba7-3f91434868b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:54.088222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-655dfd43-dbb8-4059-aba7-3f91434868b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:54.088666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-655dfd43-dbb8-4059-aba7-3f91434868b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:54.089460 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-655dfd43-dbb8-4059-aba7-3f91434868b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:54.089893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-655dfd43-dbb8-4059-aba7-3f91434868b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:54.104671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:42:54.232056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-655dfd43-dbb8-4059-aba7-3f91434868b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:54.242195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-655dfd43-dbb8-4059-aba7-3f91434868b3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172679 May 25 16:42:54.242852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1088/2154] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:54 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:54.449187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:54.449946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:54.458139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:54.458701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:54.459680 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:54.460219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:54.531700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:54.535192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8e176f32-254e-426b-8868-463dba6797d3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:54.535809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8e176f32-254e-426b-8868-463dba6797d3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:54.549763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e176f32-254e-426b-8868-463dba6797d3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:54.550210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e176f32-254e-426b-8868-463dba6797d3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:54.550371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:54.550975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e176f32-254e-426b-8868-463dba6797d3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:54.551403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8e176f32-254e-426b-8868-463dba6797d3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:54.551732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:54.552864 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dd454ec8-96d3-4bf1-ae40-fdc3f11523f8 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.109518 May 25 16:42:54.553415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1067/2155] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:54.614783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8e176f32-254e-426b-8868-463dba6797d3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:54.623388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8e176f32-254e-426b-8868-463dba6797d3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093218 May 25 16:42:54.623908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1089/2156] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:54 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:42:54.866266 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2553c5ad-e6d3-4470-b056-d466e75dd76b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:54.867298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2553c5ad-e6d3-4470-b056-d466e75dd76b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:54.874295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2553c5ad-e6d3-4470-b056-d466e75dd76b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:54.874676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2553c5ad-e6d3-4470-b056-d466e75dd76b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:54.875481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2553c5ad-e6d3-4470-b056-d466e75dd76b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:54.875913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2553c5ad-e6d3-4470-b056-d466e75dd76b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:55.014320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2553c5ad-e6d3-4470-b056-d466e75dd76b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:55.022384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2553c5ad-e6d3-4470-b056-d466e75dd76b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.162570 May 25 16:42:55.022953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1068/2157] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:54 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:55.265179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8ed6d4ec-f8bf-4464-8815-ae337790dadc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:55.266064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8ed6d4ec-f8bf-4464-8815-ae337790dadc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:55.276342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ed6d4ec-f8bf-4464-8815-ae337790dadc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:55.277077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ed6d4ec-f8bf-4464-8815-ae337790dadc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:55.278310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ed6d4ec-f8bf-4464-8815-ae337790dadc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:55.279054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8ed6d4ec-f8bf-4464-8815-ae337790dadc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:55.423913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8ed6d4ec-f8bf-4464-8815-ae337790dadc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:55.436387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8ed6d4ec-f8bf-4464-8815-ae337790dadc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.177894 May 25 16:42:55.437254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1090/2158] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:55 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:55.574055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:55.575015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:55.586060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:55.586826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:55.587630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:55.588065 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:55.640188 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-11452d4b-f088-48a1-9c03-959b6b2b7ccd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:55.640862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-11452d4b-f088-48a1-9c03-959b6b2b7ccd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:55.647864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-11452d4b-f088-48a1-9c03-959b6b2b7ccd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:55.648683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-11452d4b-f088-48a1-9c03-959b6b2b7ccd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:55.648833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:55.649644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-11452d4b-f088-48a1-9c03-959b6b2b7ccd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:55.650148 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-11452d4b-f088-48a1-9c03-959b6b2b7ccd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:55.657584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:55.659096 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:55.660257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2f84d646-3bae-44b8-94cc-5cba7a130d5b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.092784 May 25 16:42:55.660815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1069/2159] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:55 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:55.712428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-11452d4b-f088-48a1-9c03-959b6b2b7ccd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:55.720705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-11452d4b-f088-48a1-9c03-959b6b2b7ccd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084441 May 25 16:42:55.721282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1091/2160] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:55 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:56.044569 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-43f3b5ce-e3a4-4ae6-9a60-62cf39d32ef0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:56.045574 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-43f3b5ce-e3a4-4ae6-9a60-62cf39d32ef0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:56.055895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43f3b5ce-e3a4-4ae6-9a60-62cf39d32ef0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:56.056967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43f3b5ce-e3a4-4ae6-9a60-62cf39d32ef0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:56.060445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43f3b5ce-e3a4-4ae6-9a60-62cf39d32ef0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:56.061437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-43f3b5ce-e3a4-4ae6-9a60-62cf39d32ef0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:56.201219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-43f3b5ce-e3a4-4ae6-9a60-62cf39d32ef0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:56.214558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-43f3b5ce-e3a4-4ae6-9a60-62cf39d32ef0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.176668 May 25 16:42:56.215421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1070/2161] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:56 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:56.457606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-126bd623-438c-4f55-b3a9-c1b9caa65b04 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:56.458377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-126bd623-438c-4f55-b3a9-c1b9caa65b04 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:56.466611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-126bd623-438c-4f55-b3a9-c1b9caa65b04 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:56.467140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-126bd623-438c-4f55-b3a9-c1b9caa65b04 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:56.468068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-126bd623-438c-4f55-b3a9-c1b9caa65b04 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:56.468579 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-126bd623-438c-4f55-b3a9-c1b9caa65b04 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:56.606585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-126bd623-438c-4f55-b3a9-c1b9caa65b04 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:56.615005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-126bd623-438c-4f55-b3a9-c1b9caa65b04 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.163877 May 25 16:42:56.615619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1092/2162] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:56 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:56.682885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:56.683774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:56.694636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:56.695457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:56.696552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:56.696983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:56.736845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-23f0283c-5803-4653-a8b9-e0c035f2aed7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:56.737449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-23f0283c-5803-4653-a8b9-e0c035f2aed7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:56.745019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23f0283c-5803-4653-a8b9-e0c035f2aed7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:56.745462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23f0283c-5803-4653-a8b9-e0c035f2aed7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:56.746223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23f0283c-5803-4653-a8b9-e0c035f2aed7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:56.746651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23f0283c-5803-4653-a8b9-e0c035f2aed7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:56.760906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:56.769753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:56.771281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:56.772427 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-00e4baa8-9325-4d92-aef4-10ba4e345c54 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.096235 May 25 16:42:56.772990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1071/2163] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:56 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:56.819700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-23f0283c-5803-4653-a8b9-e0c035f2aed7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:56.827747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-23f0283c-5803-4653-a8b9-e0c035f2aed7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.094801 May 25 16:42:56.828349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1093/2164] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:56 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:57.236557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7ac0d9d8-2802-44a6-a5c9-2bba134372f2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:57.237560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7ac0d9d8-2802-44a6-a5c9-2bba134372f2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:57.246028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ac0d9d8-2802-44a6-a5c9-2bba134372f2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:57.246482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ac0d9d8-2802-44a6-a5c9-2bba134372f2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:57.247272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ac0d9d8-2802-44a6-a5c9-2bba134372f2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:57.247714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7ac0d9d8-2802-44a6-a5c9-2bba134372f2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:57.397398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7ac0d9d8-2802-44a6-a5c9-2bba134372f2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:57.410004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7ac0d9d8-2802-44a6-a5c9-2bba134372f2 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.179694 May 25 16:42:57.410770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1072/2165] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:57 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:57.636208 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2f147792-8fc3-41c1-a4ad-08333c5e086c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:57.636915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2f147792-8fc3-41c1-a4ad-08333c5e086c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:57.645324 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f147792-8fc3-41c1-a4ad-08333c5e086c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:57.645927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f147792-8fc3-41c1-a4ad-08333c5e086c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:57.646925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f147792-8fc3-41c1-a4ad-08333c5e086c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:57.647559 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f147792-8fc3-41c1-a4ad-08333c5e086c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:57.787579 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2f147792-8fc3-41c1-a4ad-08333c5e086c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:57.790627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:57.791230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:57.796110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2f147792-8fc3-41c1-a4ad-08333c5e086c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.166025 May 25 16:42:57.796739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1094/2166] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:57 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:57.797970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:57.798415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:57.799194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:57.799626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:57.844493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-656a6502-5178-4ab9-bf1e-c5b58c69e2f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:57.845099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-656a6502-5178-4ab9-bf1e-c5b58c69e2f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:57.851522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-656a6502-5178-4ab9-bf1e-c5b58c69e2f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:57.851975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-656a6502-5178-4ab9-bf1e-c5b58c69e2f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:57.852737 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-656a6502-5178-4ab9-bf1e-c5b58c69e2f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:57.853170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-656a6502-5178-4ab9-bf1e-c5b58c69e2f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:57.860450 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:57.869156 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:57.870656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:57.871831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-56cc814d-9adc-4e46-813f-202077ee8150 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.085741 May 25 16:42:57.882248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1073/2167] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:42:57.914655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-656a6502-5178-4ab9-bf1e-c5b58c69e2f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:57.922902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-656a6502-5178-4ab9-bf1e-c5b58c69e2f3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.082309 May 25 16:42:57.923463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1095/2168] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:57 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:58.431510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a12595f-254c-49a1-8478-1c8c813e6e64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:58.432395 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7a12595f-254c-49a1-8478-1c8c813e6e64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:58.441821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a12595f-254c-49a1-8478-1c8c813e6e64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:58.442396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a12595f-254c-49a1-8478-1c8c813e6e64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:58.443278 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a12595f-254c-49a1-8478-1c8c813e6e64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:58.443626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a12595f-254c-49a1-8478-1c8c813e6e64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:58.576770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7a12595f-254c-49a1-8478-1c8c813e6e64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:58.584929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7a12595f-254c-49a1-8478-1c8c813e6e64 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.159132 May 25 16:42:58.585474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1074/2169] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:58 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:58.816800 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f25f877c-33ac-49a4-a763-2011840b3645 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:58.817802 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f25f877c-33ac-49a4-a763-2011840b3645 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:58.828818 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f25f877c-33ac-49a4-a763-2011840b3645 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:58.829567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f25f877c-33ac-49a4-a763-2011840b3645 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:58.830830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f25f877c-33ac-49a4-a763-2011840b3645 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:58.831616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f25f877c-33ac-49a4-a763-2011840b3645 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:58.890095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:58.890700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:58.897019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:58.897476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:58.898227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:58.898663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:58.967803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:58.976389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:58.977944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:58.979085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f629df54-db3b-499c-8959-051cc11a1d6e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.092980 May 25 16:42:58.979686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1075/2170] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:58 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:42:58.984404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e8afe00a-75b2-4382-b785-1db23fb1fd7c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:58.985005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e8afe00a-75b2-4382-b785-1db23fb1fd7c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:58.989223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f25f877c-33ac-49a4-a763-2011840b3645 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:58.991492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8afe00a-75b2-4382-b785-1db23fb1fd7c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:58.991948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8afe00a-75b2-4382-b785-1db23fb1fd7c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:58.992755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8afe00a-75b2-4382-b785-1db23fb1fd7c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:58.993193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e8afe00a-75b2-4382-b785-1db23fb1fd7c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:58.998223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f25f877c-33ac-49a4-a763-2011840b3645 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.188005 May 25 16:42:58.998886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1096/2171] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:58 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:59.054106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e8afe00a-75b2-4382-b785-1db23fb1fd7c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:59.062557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e8afe00a-75b2-4382-b785-1db23fb1fd7c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.081950 May 25 16:42:59.063125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1076/2172] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:58 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:42:59.603133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-03c515ab-c4ac-46f7-87a8-3b06dc047bad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:42:59.603809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-03c515ab-c4ac-46f7-87a8-3b06dc047bad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:42:59.610965 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03c515ab-c4ac-46f7-87a8-3b06dc047bad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:59.611490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03c515ab-c4ac-46f7-87a8-3b06dc047bad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:59.612329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03c515ab-c4ac-46f7-87a8-3b06dc047bad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:42:59.612807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-03c515ab-c4ac-46f7-87a8-3b06dc047bad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:42:59.753509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-03c515ab-c4ac-46f7-87a8-3b06dc047bad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:42:59.762072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-03c515ab-c4ac-46f7-87a8-3b06dc047bad tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.163446 May 25 16:42:59.762667 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1097/2173] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:42:59 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:00.001579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:00.002430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:00.011677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:00.012102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:00.012899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:00.013396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:00.015794 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e1da10ba-461b-4af9-95c4-7023e7b90af8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:00.016404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e1da10ba-461b-4af9-95c4-7023e7b90af8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:00.023143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1da10ba-461b-4af9-95c4-7023e7b90af8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:00.023638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1da10ba-461b-4af9-95c4-7023e7b90af8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:00.024723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1da10ba-461b-4af9-95c4-7023e7b90af8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:00.025133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1da10ba-461b-4af9-95c4-7023e7b90af8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:00.093137 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:00.101547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:00.103338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:00.104463 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d62730b6-f192-44a6-92a2-d17df2857db9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.109835 May 25 16:43:00.105003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1077/2174] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:42:59 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:00.109441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bd4bd325-cb47-4b2c-be13-72a5dbcb580b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:00.110238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bd4bd325-cb47-4b2c-be13-72a5dbcb580b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:00.116244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd4bd325-cb47-4b2c-be13-72a5dbcb580b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:00.116682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd4bd325-cb47-4b2c-be13-72a5dbcb580b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:00.117430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd4bd325-cb47-4b2c-be13-72a5dbcb580b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:00.118143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bd4bd325-cb47-4b2c-be13-72a5dbcb580b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:00.188468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bd4bd325-cb47-4b2c-be13-72a5dbcb580b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:00.196704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bd4bd325-cb47-4b2c-be13-72a5dbcb580b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090507 May 25 16:43:00.197434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1078/2175] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:00 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:00.204941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e1da10ba-461b-4af9-95c4-7023e7b90af8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:00.215219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e1da10ba-461b-4af9-95c4-7023e7b90af8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.203363 May 25 16:43:00.215785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1098/2176] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:00 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:00.783697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c830915b-6955-4ef9-a073-331b83893cfd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:00.784696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c830915b-6955-4ef9-a073-331b83893cfd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:00.795424 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c830915b-6955-4ef9-a073-331b83893cfd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:00.796176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c830915b-6955-4ef9-a073-331b83893cfd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:00.797426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c830915b-6955-4ef9-a073-331b83893cfd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:00.798146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c830915b-6955-4ef9-a073-331b83893cfd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:00.802365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:00.937219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c830915b-6955-4ef9-a073-331b83893cfd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:00.950241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c830915b-6955-4ef9-a073-331b83893cfd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.172807 May 25 16:43:00.950987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1079/2177] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:00 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:01.123532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:01.124157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:01.131298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:01.131751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:01.132512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:01.132947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:01.208262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:01.213602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dc20bfbc-db12-4aa4-b3fa-60175f024f7e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:01.214207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dc20bfbc-db12-4aa4-b3fa-60175f024f7e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:01.217181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:01.218830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:01.220099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-58e6914b-7f4f-4219-a461-16e369f18c62 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.100717 May 25 16:43:01.220651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1099/2178] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:01 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:01.221207 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc20bfbc-db12-4aa4-b3fa-60175f024f7e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:01.221664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc20bfbc-db12-4aa4-b3fa-60175f024f7e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:01.222422 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc20bfbc-db12-4aa4-b3fa-60175f024f7e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:01.222867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dc20bfbc-db12-4aa4-b3fa-60175f024f7e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:01.232058 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-edc42e74-7557-4bc4-9e08-8533b3e7f7be tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:01.232659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-edc42e74-7557-4bc4-9e08-8533b3e7f7be tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:01.238995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edc42e74-7557-4bc4-9e08-8533b3e7f7be tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:01.239514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edc42e74-7557-4bc4-9e08-8533b3e7f7be tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:01.240259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edc42e74-7557-4bc4-9e08-8533b3e7f7be tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:01.240704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edc42e74-7557-4bc4-9e08-8533b3e7f7be tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:01.290612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dc20bfbc-db12-4aa4-b3fa-60175f024f7e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:01.299090 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dc20bfbc-db12-4aa4-b3fa-60175f024f7e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089291 May 25 16:43:01.299694 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1080/2179] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:01 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:01.382443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-edc42e74-7557-4bc4-9e08-8533b3e7f7be tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:01.390798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-edc42e74-7557-4bc4-9e08-8533b3e7f7be tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162384 May 25 16:43:01.391379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1100/2180] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:01 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:43:01.976066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ee5fa3dc-ec55-480b-a2f7-fde57fb9dadd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:01.977174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ee5fa3dc-ec55-480b-a2f7-fde57fb9dadd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:01.983709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee5fa3dc-ec55-480b-a2f7-fde57fb9dadd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:01.984164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee5fa3dc-ec55-480b-a2f7-fde57fb9dadd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:01.984925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee5fa3dc-ec55-480b-a2f7-fde57fb9dadd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:01.985358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee5fa3dc-ec55-480b-a2f7-fde57fb9dadd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:02.123462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ee5fa3dc-ec55-480b-a2f7-fde57fb9dadd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:02.132003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ee5fa3dc-ec55-480b-a2f7-fde57fb9dadd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.162546 May 25 16:43:02.132597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1081/2181] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:01 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:02.242740 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:02.243662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:02.254811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:02.255620 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:02.256868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:02.257588 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:02.315428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-563278de-ccdb-4f40-97db-fb93a194c6ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:02.316016 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-563278de-ccdb-4f40-97db-fb93a194c6ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:02.322888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-563278de-ccdb-4f40-97db-fb93a194c6ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:02.323382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-563278de-ccdb-4f40-97db-fb93a194c6ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:02.324150 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-563278de-ccdb-4f40-97db-fb93a194c6ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:02.324584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-563278de-ccdb-4f40-97db-fb93a194c6ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:02.325112 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:02.333932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:02.335449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:02.336973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-edbde810-8efb-45ed-8e9e-d246bcf68809 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.100540 May 25 16:43:02.337521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1101/2182] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:02 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:02.385648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-563278de-ccdb-4f40-97db-fb93a194c6ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:02.393934 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-563278de-ccdb-4f40-97db-fb93a194c6ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.082532 May 25 16:43:02.394724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1082/2183] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:02 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:02.411378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-69f10dad-c89a-40ff-938e-5ed75c32fa97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:02.412275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-69f10dad-c89a-40ff-938e-5ed75c32fa97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:02.421444 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69f10dad-c89a-40ff-938e-5ed75c32fa97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:02.422082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69f10dad-c89a-40ff-938e-5ed75c32fa97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:02.423262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69f10dad-c89a-40ff-938e-5ed75c32fa97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:02.423868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-69f10dad-c89a-40ff-938e-5ed75c32fa97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:02.559592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-69f10dad-c89a-40ff-938e-5ed75c32fa97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:02.567690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-69f10dad-c89a-40ff-938e-5ed75c32fa97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162831 May 25 16:43:02.568263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1102/2184] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:02 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:03.157895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-666e5a68-5497-4c0f-994b-7a3d1e03f3d3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:03.158854 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-666e5a68-5497-4c0f-994b-7a3d1e03f3d3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:03.171425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-666e5a68-5497-4c0f-994b-7a3d1e03f3d3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:03.172167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-666e5a68-5497-4c0f-994b-7a3d1e03f3d3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:03.173435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-666e5a68-5497-4c0f-994b-7a3d1e03f3d3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:03.174062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-666e5a68-5497-4c0f-994b-7a3d1e03f3d3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:03.313153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-666e5a68-5497-4c0f-994b-7a3d1e03f3d3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:03.322286 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-666e5a68-5497-4c0f-994b-7a3d1e03f3d3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.175433 May 25 16:43:03.323910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1083/2185] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:03 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:03.354063 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:03.354703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:03.362055 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:03.362536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:03.363373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:03.363806 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:03.410575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8e2abff4-1d24-44ec-9f6d-740d1f5a6505 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:03.411212 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8e2abff4-1d24-44ec-9f6d-740d1f5a6505 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:03.417794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8e2abff4-1d24-44ec-9f6d-740d1f5a6505 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:03.418249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8e2abff4-1d24-44ec-9f6d-740d1f5a6505 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:03.419044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8e2abff4-1d24-44ec-9f6d-740d1f5a6505 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:03.419506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8e2abff4-1d24-44ec-9f6d-740d1f5a6505 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:03.426620 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:03.438844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:03.440377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:03.441524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-83f37546-9a07-44ce-8a86-a7c38df2ac4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.091470 May 25 16:43:03.442097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1103/2186] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:03 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:03.488839 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8e2abff4-1d24-44ec-9f6d-740d1f5a6505 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:03.497062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8e2abff4-1d24-44ec-9f6d-740d1f5a6505 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090713 May 25 16:43:03.497633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1084/2187] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:03 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:03.589050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b5a9dee1-debe-432b-ac0a-85837bc40c93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:03.589977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b5a9dee1-debe-432b-ac0a-85837bc40c93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:03.600297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5a9dee1-debe-432b-ac0a-85837bc40c93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:03.601051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5a9dee1-debe-432b-ac0a-85837bc40c93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:03.602324 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5a9dee1-debe-432b-ac0a-85837bc40c93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:03.602756 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5a9dee1-debe-432b-ac0a-85837bc40c93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:03.744407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b5a9dee1-debe-432b-ac0a-85837bc40c93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:03.752596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b5a9dee1-debe-432b-ac0a-85837bc40c93 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.170126 May 25 16:43:03.753142 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1104/2188] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:03 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:04.348836 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-079ba2ec-53b7-41f8-8d5d-83ddbfbc5ccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:04.349792 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-079ba2ec-53b7-41f8-8d5d-83ddbfbc5ccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:04.357886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-079ba2ec-53b7-41f8-8d5d-83ddbfbc5ccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:04.358333 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-079ba2ec-53b7-41f8-8d5d-83ddbfbc5ccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:04.359074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-079ba2ec-53b7-41f8-8d5d-83ddbfbc5ccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:04.359556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-079ba2ec-53b7-41f8-8d5d-83ddbfbc5ccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:04.459864 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:04.460431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:04.467345 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:04.467796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:04.468558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:04.468994 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:04.472959 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:04.492103 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-079ba2ec-53b7-41f8-8d5d-83ddbfbc5ccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:04.500433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-079ba2ec-53b7-41f8-8d5d-83ddbfbc5ccc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.158132 May 25 16:43:04.501040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1085/2189] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:04 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:04.513770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fffeb468-52a5-4aae-b698-70aeb016677a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:04.514505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fffeb468-52a5-4aae-b698-70aeb016677a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:04.520738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fffeb468-52a5-4aae-b698-70aeb016677a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:04.521202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fffeb468-52a5-4aae-b698-70aeb016677a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:04.521953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fffeb468-52a5-4aae-b698-70aeb016677a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:04.522383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fffeb468-52a5-4aae-b698-70aeb016677a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:04.530262 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:04.538796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:04.540678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:04.541822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e506520c-50af-4c4d-99b5-1d7204be30eb tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.085964 May 25 16:43:04.542363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1105/2190] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:04 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:04.583519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fffeb468-52a5-4aae-b698-70aeb016677a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:04.591729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fffeb468-52a5-4aae-b698-70aeb016677a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.081810 May 25 16:43:04.592296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1086/2191] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:04 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:43:04.773631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8672ed88-a490-4661-bea2-d1a717cf2991 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:04.774634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8672ed88-a490-4661-bea2-d1a717cf2991 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:04.781409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8672ed88-a490-4661-bea2-d1a717cf2991 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:04.781861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8672ed88-a490-4661-bea2-d1a717cf2991 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:04.782682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8672ed88-a490-4661-bea2-d1a717cf2991 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:04.783137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8672ed88-a490-4661-bea2-d1a717cf2991 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:04.920344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8672ed88-a490-4661-bea2-d1a717cf2991 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:04.928499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8672ed88-a490-4661-bea2-d1a717cf2991 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.161387 May 25 16:43:04.929071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1106/2192] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:04 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:05.521091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9903b696-d24b-481a-8cdc-52c4bb226a6f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:05.521946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9903b696-d24b-481a-8cdc-52c4bb226a6f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:05.529978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9903b696-d24b-481a-8cdc-52c4bb226a6f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:05.530424 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9903b696-d24b-481a-8cdc-52c4bb226a6f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:05.531185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9903b696-d24b-481a-8cdc-52c4bb226a6f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:05.531649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9903b696-d24b-481a-8cdc-52c4bb226a6f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:05.560013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:05.560587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:05.567557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:05.568005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:05.568763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:05.569170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:05.644132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:05.654490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:05.656705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:05.658326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7eb1e2ae-2eec-4d82-bae7-e942038a049b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.102060 May 25 16:43:05.659166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1107/2193] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:05 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:05.663568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-63a50805-328d-42bd-a705-e7683faa2a10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:05.664488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-63a50805-328d-42bd-a705-e7683faa2a10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:05.671031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63a50805-328d-42bd-a705-e7683faa2a10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:05.671512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63a50805-328d-42bd-a705-e7683faa2a10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:05.672270 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63a50805-328d-42bd-a705-e7683faa2a10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:05.672704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-63a50805-328d-42bd-a705-e7683faa2a10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:05.683542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9903b696-d24b-481a-8cdc-52c4bb226a6f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:05.691729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9903b696-d24b-481a-8cdc-52c4bb226a6f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.177217 May 25 16:43:05.692295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1087/2194] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:05 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:05.946653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4d9f32a1-062c-4330-ba68-464faef5094c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:05.947272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4d9f32a1-062c-4330-ba68-464faef5094c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:05.954157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d9f32a1-062c-4330-ba68-464faef5094c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:05.954603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d9f32a1-062c-4330-ba68-464faef5094c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:05.955438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d9f32a1-062c-4330-ba68-464faef5094c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:05.955871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d9f32a1-062c-4330-ba68-464faef5094c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:05.980751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-63a50805-328d-42bd-a705-e7683faa2a10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:05.988895 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-63a50805-328d-42bd-a705-e7683faa2a10 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.328766 May 25 16:43:05.989479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1108/2195] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:05 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:06.087818 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4d9f32a1-062c-4330-ba68-464faef5094c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:06.095852 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4d9f32a1-062c-4330-ba68-464faef5094c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.153530 May 25 16:43:06.096420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1088/2196] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:05 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:06.681242 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:06.682132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:06.693064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:06.693803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:06.695059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:06.695815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:06.709314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-46372ff7-ba4b-41f7-9d60-004feebb510b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:06.709954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-46372ff7-ba4b-41f7-9d60-004feebb510b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:06.716687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46372ff7-ba4b-41f7-9d60-004feebb510b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:06.717134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46372ff7-ba4b-41f7-9d60-004feebb510b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:06.717885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46372ff7-ba4b-41f7-9d60-004feebb510b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:06.718313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46372ff7-ba4b-41f7-9d60-004feebb510b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:06.769654 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:06.778920 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:06.780447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:06.781592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ee6a0655-8fbc-4dbe-84b8-19e4812a3bac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.107224 May 25 16:43:06.782146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1109/2197] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:06 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:06.857961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-46372ff7-ba4b-41f7-9d60-004feebb510b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:06.867004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-46372ff7-ba4b-41f7-9d60-004feebb510b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.161516 May 25 16:43:06.867607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1089/2198] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:06 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:07.006490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a1574086-d9bb-46be-a108-ea72dbd23c08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:07.007097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a1574086-d9bb-46be-a108-ea72dbd23c08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:07.013249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1574086-d9bb-46be-a108-ea72dbd23c08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:07.013699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1574086-d9bb-46be-a108-ea72dbd23c08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:07.014466 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1574086-d9bb-46be-a108-ea72dbd23c08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:07.014881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1574086-d9bb-46be-a108-ea72dbd23c08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:07.076186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a1574086-d9bb-46be-a108-ea72dbd23c08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:07.084361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a1574086-d9bb-46be-a108-ea72dbd23c08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.081791 May 25 16:43:07.084926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1110/2199] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:07 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:07.116181 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c64dea4e-7f63-4422-9ec4-25e360d44c37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:07.117206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c64dea4e-7f63-4422-9ec4-25e360d44c37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:07.123905 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c64dea4e-7f63-4422-9ec4-25e360d44c37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:07.124349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c64dea4e-7f63-4422-9ec4-25e360d44c37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:07.125093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c64dea4e-7f63-4422-9ec4-25e360d44c37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:07.125519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c64dea4e-7f63-4422-9ec4-25e360d44c37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:07.262923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c64dea4e-7f63-4422-9ec4-25e360d44c37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:07.275250 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c64dea4e-7f63-4422-9ec4-25e360d44c37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.165224 May 25 16:43:07.276008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1090/2200] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:07 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:07.801556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:07.802257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:07.809702 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:07.810154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:07.810902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:07.811369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:07.883805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:07.884538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d62eb84c-1b23-4e68-8baf-a904a73cdc72 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:07.885141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d62eb84c-1b23-4e68-8baf-a904a73cdc72 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:07.891634 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d62eb84c-1b23-4e68-8baf-a904a73cdc72 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:07.892083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d62eb84c-1b23-4e68-8baf-a904a73cdc72 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:07.892831 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d62eb84c-1b23-4e68-8baf-a904a73cdc72 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:07.893068 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:07.893243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d62eb84c-1b23-4e68-8baf-a904a73cdc72 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:07.894575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:07.895737 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a8cccb63-ebad-4f1f-aceb-72bf0d057765 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098852 May 25 16:43:07.896266 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1111/2201] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:07 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:08.028851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d62eb84c-1b23-4e68-8baf-a904a73cdc72 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:08.037017 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d62eb84c-1b23-4e68-8baf-a904a73cdc72 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.156412 May 25 16:43:08.037581 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1091/2202] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:07 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:43:08.101154 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9dea76d6-f256-451c-a427-a402c186425d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:08.101762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9dea76d6-f256-451c-a427-a402c186425d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:08.108206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9dea76d6-f256-451c-a427-a402c186425d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:08.108641 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9dea76d6-f256-451c-a427-a402c186425d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:08.109390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9dea76d6-f256-451c-a427-a402c186425d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:08.109822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9dea76d6-f256-451c-a427-a402c186425d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:08.173673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9dea76d6-f256-451c-a427-a402c186425d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:08.186747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9dea76d6-f256-451c-a427-a402c186425d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089224 May 25 16:43:08.187689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1112/2203] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:08 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:08.302077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-41efc56b-f100-4993-9a86-fab55a1f0e32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:08.303079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-41efc56b-f100-4993-9a86-fab55a1f0e32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:08.310524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41efc56b-f100-4993-9a86-fab55a1f0e32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:08.310974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41efc56b-f100-4993-9a86-fab55a1f0e32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:08.311758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41efc56b-f100-4993-9a86-fab55a1f0e32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:08.312190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41efc56b-f100-4993-9a86-fab55a1f0e32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:08.442927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-41efc56b-f100-4993-9a86-fab55a1f0e32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:08.451293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-41efc56b-f100-4993-9a86-fab55a1f0e32 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.155496 May 25 16:43:08.452030 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1092/2204] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:08 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:08.917286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:08.918013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:08.925674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:08.926179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:08.927023 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:08.927538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:08.990451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:08.999130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:09.000672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:09.001810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1a54f51f-0d8f-4b56-b810-4f2f2f4b683a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.090598 May 25 16:43:09.002384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1113/2205] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:08 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:09.056080 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7879ceaf-109f-4f94-85f7-ba290f13fd63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:09.056686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7879ceaf-109f-4f94-85f7-ba290f13fd63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:09.064120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7879ceaf-109f-4f94-85f7-ba290f13fd63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:09.064571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7879ceaf-109f-4f94-85f7-ba290f13fd63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:09.065327 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7879ceaf-109f-4f94-85f7-ba290f13fd63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:09.065757 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7879ceaf-109f-4f94-85f7-ba290f13fd63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:09.194573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7879ceaf-109f-4f94-85f7-ba290f13fd63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:09.203038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7879ceaf-109f-4f94-85f7-ba290f13fd63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.150890 May 25 16:43:09.203666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1093/2206] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:09 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:09.207708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4832c917-d527-40c6-abc3-93a3d7b3a2fe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:09.208318 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4832c917-d527-40c6-abc3-93a3d7b3a2fe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:09.215073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4832c917-d527-40c6-abc3-93a3d7b3a2fe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:09.215745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4832c917-d527-40c6-abc3-93a3d7b3a2fe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:09.216565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4832c917-d527-40c6-abc3-93a3d7b3a2fe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:09.217119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4832c917-d527-40c6-abc3-93a3d7b3a2fe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:09.286594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4832c917-d527-40c6-abc3-93a3d7b3a2fe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:09.300370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4832c917-d527-40c6-abc3-93a3d7b3a2fe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.096318 May 25 16:43:09.301265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1114/2207] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:09 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:09.473285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c513e1b9-1cac-4faa-8adc-ad44ebe02759 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:09.474152 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c513e1b9-1cac-4faa-8adc-ad44ebe02759 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:09.483316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c513e1b9-1cac-4faa-8adc-ad44ebe02759 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:09.483768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c513e1b9-1cac-4faa-8adc-ad44ebe02759 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:09.484532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c513e1b9-1cac-4faa-8adc-ad44ebe02759 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:09.484948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c513e1b9-1cac-4faa-8adc-ad44ebe02759 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:09.623140 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c513e1b9-1cac-4faa-8adc-ad44ebe02759 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:09.631815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c513e1b9-1cac-4faa-8adc-ad44ebe02759 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.165227 May 25 16:43:09.632381 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1094/2208] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:09 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:10.024499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:10.025130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:10.032149 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:10.032595 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:10.033354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:10.033785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:10.096889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:10.105695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:10.107194 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:10.108351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bc3a7b39-8228-4970-9d3c-bc1ee3723af5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.090585 May 25 16:43:10.108919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1115/2209] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:10 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:10.225143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-39e58468-142c-4c17-9bdc-0fb675083468 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:10.226153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-39e58468-142c-4c17-9bdc-0fb675083468 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:10.229032 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:10.234813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39e58468-142c-4c17-9bdc-0fb675083468 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:10.235297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39e58468-142c-4c17-9bdc-0fb675083468 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:10.236058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39e58468-142c-4c17-9bdc-0fb675083468 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:10.236490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-39e58468-142c-4c17-9bdc-0fb675083468 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:10.317268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7f8fa5e1-99fe-4e31-8fc2-c6321b29a47d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:10.317787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7f8fa5e1-99fe-4e31-8fc2-c6321b29a47d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:10.324323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f8fa5e1-99fe-4e31-8fc2-c6321b29a47d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:10.324780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f8fa5e1-99fe-4e31-8fc2-c6321b29a47d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:10.325546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f8fa5e1-99fe-4e31-8fc2-c6321b29a47d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:10.325985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7f8fa5e1-99fe-4e31-8fc2-c6321b29a47d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:10.396074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7f8fa5e1-99fe-4e31-8fc2-c6321b29a47d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:10.399647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-39e58468-142c-4c17-9bdc-0fb675083468 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:10.401821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:10.404426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7f8fa5e1-99fe-4e31-8fc2-c6321b29a47d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091114 May 25 16:43:10.405671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1116/2210] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:10 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:43:10.409370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-39e58468-142c-4c17-9bdc-0fb675083468 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.190745 May 25 16:43:10.409935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1095/2211] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:10 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:10.653723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1c3b7633-5d7c-43a4-a9b4-9bdfe5dc2e84 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:10.654598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1c3b7633-5d7c-43a4-a9b4-9bdfe5dc2e84 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:10.665113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c3b7633-5d7c-43a4-a9b4-9bdfe5dc2e84 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:10.665851 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c3b7633-5d7c-43a4-a9b4-9bdfe5dc2e84 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:10.667181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c3b7633-5d7c-43a4-a9b4-9bdfe5dc2e84 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:10.667912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1c3b7633-5d7c-43a4-a9b4-9bdfe5dc2e84 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:10.812708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1c3b7633-5d7c-43a4-a9b4-9bdfe5dc2e84 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:10.823222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1c3b7633-5d7c-43a4-a9b4-9bdfe5dc2e84 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.176392 May 25 16:43:10.824014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1117/2212] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:10 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:11.126867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:11.127506 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:11.134347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:11.134805 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:11.135605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:11.136074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:11.202577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:11.215641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:11.217929 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:11.219399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-63f4b8e2-8d56-40bd-9ec5-e1d8781d8e89 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.096239 May 25 16:43:11.220134 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1096/2213] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:11 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:11.425767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5ddf55bd-9e45-4782-80ac-8db0b96159bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:11.426632 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5ddf55bd-9e45-4782-80ac-8db0b96159bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:11.431636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e6539bbd-f455-4648-b87a-8bd99566654a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:11.432247 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e6539bbd-f455-4648-b87a-8bd99566654a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:11.435171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5ddf55bd-9e45-4782-80ac-8db0b96159bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:11.435644 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5ddf55bd-9e45-4782-80ac-8db0b96159bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:11.436396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5ddf55bd-9e45-4782-80ac-8db0b96159bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:11.436819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5ddf55bd-9e45-4782-80ac-8db0b96159bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:11.439058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e6539bbd-f455-4648-b87a-8bd99566654a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:11.439558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e6539bbd-f455-4648-b87a-8bd99566654a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:11.440324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e6539bbd-f455-4648-b87a-8bd99566654a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:11.440740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e6539bbd-f455-4648-b87a-8bd99566654a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:11.523258 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5ddf55bd-9e45-4782-80ac-8db0b96159bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:11.531489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5ddf55bd-9e45-4782-80ac-8db0b96159bb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.112508 May 25 16:43:11.532079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1118/2214] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:11 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:11.594004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e6539bbd-f455-4648-b87a-8bd99566654a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:11.606602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e6539bbd-f455-4648-b87a-8bd99566654a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.180859 May 25 16:43:11.607268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1097/2215] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:11 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:11.848967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-047d29a6-95b5-494a-92f3-9f2f2a6d12a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:11.850030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-047d29a6-95b5-494a-92f3-9f2f2a6d12a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:11.857628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-047d29a6-95b5-494a-92f3-9f2f2a6d12a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:11.858082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-047d29a6-95b5-494a-92f3-9f2f2a6d12a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:11.858841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-047d29a6-95b5-494a-92f3-9f2f2a6d12a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:11.859340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-047d29a6-95b5-494a-92f3-9f2f2a6d12a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:11.993469 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-047d29a6-95b5-494a-92f3-9f2f2a6d12a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:12.001628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-047d29a6-95b5-494a-92f3-9f2f2a6d12a0 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.159195 May 25 16:43:12.002205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1119/2216] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:11 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:12.244420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:12.245141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:12.253077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:12.253644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:12.254627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:12.255214 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:12.328504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:12.337295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:12.338802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:12.340034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1aa5b347-54e4-4627-b457-6965d141a958 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.102420 May 25 16:43:12.340711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1098/2217] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:12 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:12.549764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-dde8703f-b263-40ae-bbe8-281aeccb7cf8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:12.550381 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-dde8703f-b263-40ae-bbe8-281aeccb7cf8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:12.557673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dde8703f-b263-40ae-bbe8-281aeccb7cf8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:12.558119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dde8703f-b263-40ae-bbe8-281aeccb7cf8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:12.558888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dde8703f-b263-40ae-bbe8-281aeccb7cf8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:12.559353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-dde8703f-b263-40ae-bbe8-281aeccb7cf8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:12.623913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7660de20-8cea-4382-8888-3d3e64f3c6d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:12.624815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7660de20-8cea-4382-8888-3d3e64f3c6d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:12.626684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-dde8703f-b263-40ae-bbe8-281aeccb7cf8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:12.631703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7660de20-8cea-4382-8888-3d3e64f3c6d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:12.632228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7660de20-8cea-4382-8888-3d3e64f3c6d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:12.632957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7660de20-8cea-4382-8888-3d3e64f3c6d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:12.633424 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7660de20-8cea-4382-8888-3d3e64f3c6d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:12.635428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-dde8703f-b263-40ae-bbe8-281aeccb7cf8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089559 May 25 16:43:12.636045 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1120/2218] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:12 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:12.774890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7660de20-8cea-4382-8888-3d3e64f3c6d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:12.783136 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7660de20-8cea-4382-8888-3d3e64f3c6d1 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.163132 May 25 16:43:12.783732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1099/2219] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:12 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:13.028213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-531e9858-f2eb-4414-a13c-c2e3e86b6679 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:13.028926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-531e9858-f2eb-4414-a13c-c2e3e86b6679 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:13.035431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-531e9858-f2eb-4414-a13c-c2e3e86b6679 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:13.035884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-531e9858-f2eb-4414-a13c-c2e3e86b6679 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:13.036637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-531e9858-f2eb-4414-a13c-c2e3e86b6679 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:13.037074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-531e9858-f2eb-4414-a13c-c2e3e86b6679 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:13.169326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-531e9858-f2eb-4414-a13c-c2e3e86b6679 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:13.172726 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:13.183409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-531e9858-f2eb-4414-a13c-c2e3e86b6679 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162077 May 25 16:43:13.184170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1121/2220] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:13 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:13.363443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:13.364304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:13.374631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:13.375439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:13.376686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:13.377408 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:13.444290 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:13.453553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:13.455050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:13.456219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-df11c987-ed8c-4faf-86b2-3f6795a63dba tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.099527 May 25 16:43:13.456782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1100/2221] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:13 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:13.654688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8bf22c5f-03ac-44cb-95bd-9fa1dc8f6a8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:13.655432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8bf22c5f-03ac-44cb-95bd-9fa1dc8f6a8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:13.664231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bf22c5f-03ac-44cb-95bd-9fa1dc8f6a8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:13.664784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bf22c5f-03ac-44cb-95bd-9fa1dc8f6a8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:13.665717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bf22c5f-03ac-44cb-95bd-9fa1dc8f6a8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:13.666265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8bf22c5f-03ac-44cb-95bd-9fa1dc8f6a8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:13.732249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8bf22c5f-03ac-44cb-95bd-9fa1dc8f6a8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:13.740309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8bf22c5f-03ac-44cb-95bd-9fa1dc8f6a8f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091118 May 25 16:43:13.740881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1122/2222] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:13 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:13.807680 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b9bd6bd4-90ae-44a1-91f4-ef92d41a2ad7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:13.808531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b9bd6bd4-90ae-44a1-91f4-ef92d41a2ad7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:13.817881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9bd6bd4-90ae-44a1-91f4-ef92d41a2ad7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:13.818512 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9bd6bd4-90ae-44a1-91f4-ef92d41a2ad7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:13.819612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9bd6bd4-90ae-44a1-91f4-ef92d41a2ad7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:13.820222 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b9bd6bd4-90ae-44a1-91f4-ef92d41a2ad7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:13.967763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b9bd6bd4-90ae-44a1-91f4-ef92d41a2ad7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:13.975944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b9bd6bd4-90ae-44a1-91f4-ef92d41a2ad7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.173351 May 25 16:43:13.976517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1101/2223] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:13 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:14.204685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8422d0e4-88e6-46bd-a5a6-7f8619b1ffc4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:14.205585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8422d0e4-88e6-46bd-a5a6-7f8619b1ffc4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:14.216004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8422d0e4-88e6-46bd-a5a6-7f8619b1ffc4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:14.216762 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8422d0e4-88e6-46bd-a5a6-7f8619b1ffc4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:14.218028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8422d0e4-88e6-46bd-a5a6-7f8619b1ffc4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:14.218768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8422d0e4-88e6-46bd-a5a6-7f8619b1ffc4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:14.369164 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8422d0e4-88e6-46bd-a5a6-7f8619b1ffc4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:14.377581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8422d0e4-88e6-46bd-a5a6-7f8619b1ffc4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.179893 May 25 16:43:14.378152 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1123/2224] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:14 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:14.478042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:14.479102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:14.489248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:14.489997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:14.491306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:14.492029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:14.557505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:14.566317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:14.567850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:14.569015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-177ac2f9-2bac-4e6c-8c45-185e72697500 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.097606 May 25 16:43:14.569547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1102/2225] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:14 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:14.761582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cf5bfaff-eac9-41d5-84e7-7e9b6709ecad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:14.762453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cf5bfaff-eac9-41d5-84e7-7e9b6709ecad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:14.773446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf5bfaff-eac9-41d5-84e7-7e9b6709ecad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:14.774186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf5bfaff-eac9-41d5-84e7-7e9b6709ecad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:14.775484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf5bfaff-eac9-41d5-84e7-7e9b6709ecad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:14.776201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cf5bfaff-eac9-41d5-84e7-7e9b6709ecad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:14.843934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cf5bfaff-eac9-41d5-84e7-7e9b6709ecad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:14.852590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cf5bfaff-eac9-41d5-84e7-7e9b6709ecad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.097576 May 25 16:43:14.853159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1124/2226] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:14 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:14.995773 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c340ac3d-0f47-4a31-9c90-7bb73d57cf63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:14.996363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c340ac3d-0f47-4a31-9c90-7bb73d57cf63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:15.003252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c340ac3d-0f47-4a31-9c90-7bb73d57cf63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:15.003704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c340ac3d-0f47-4a31-9c90-7bb73d57cf63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:15.004459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c340ac3d-0f47-4a31-9c90-7bb73d57cf63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:15.004891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c340ac3d-0f47-4a31-9c90-7bb73d57cf63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:15.149347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c340ac3d-0f47-4a31-9c90-7bb73d57cf63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:15.157904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c340ac3d-0f47-4a31-9c90-7bb73d57cf63 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.166989 May 25 16:43:15.158482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1103/2227] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:14 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:15.398441 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-253156f1-e79c-45a3-8d99-0161d0c41e2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:15.399407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-253156f1-e79c-45a3-8d99-0161d0c41e2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:15.409604 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-253156f1-e79c-45a3-8d99-0161d0c41e2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:15.410370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-253156f1-e79c-45a3-8d99-0161d0c41e2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:15.411656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-253156f1-e79c-45a3-8d99-0161d0c41e2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:15.412376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-253156f1-e79c-45a3-8d99-0161d0c41e2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:15.559432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-253156f1-e79c-45a3-8d99-0161d0c41e2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:15.567553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-253156f1-e79c-45a3-8d99-0161d0c41e2f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.175601 May 25 16:43:15.568121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1125/2228] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:15 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:15.585920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:15.586561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:15.592728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:15.593182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:15.593936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:15.594372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:15.656480 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:15.665448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:15.666940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:15.668101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5ece2fbd-f367-4eab-9e68-f5395db41404 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.086079 May 25 16:43:15.668675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1104/2229] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:15 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:15.872104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-116625e8-1170-4560-8479-41d08af76273 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:15.872790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-116625e8-1170-4560-8479-41d08af76273 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:15.875497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:15.882038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-116625e8-1170-4560-8479-41d08af76273 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:15.882567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-116625e8-1170-4560-8479-41d08af76273 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:15.883483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-116625e8-1170-4560-8479-41d08af76273 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:15.883996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-116625e8-1170-4560-8479-41d08af76273 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:15.947656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-116625e8-1170-4560-8479-41d08af76273 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:15.956106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-116625e8-1170-4560-8479-41d08af76273 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089039 May 25 16:43:15.956682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1126/2230] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:15 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:16.175907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b421e411-a4d2-4153-bdbc-e87b00798940 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:16.176515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b421e411-a4d2-4153-bdbc-e87b00798940 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:16.183557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b421e411-a4d2-4153-bdbc-e87b00798940 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:16.184010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b421e411-a4d2-4153-bdbc-e87b00798940 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:16.184765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b421e411-a4d2-4153-bdbc-e87b00798940 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:16.185193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b421e411-a4d2-4153-bdbc-e87b00798940 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:16.322779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b421e411-a4d2-4153-bdbc-e87b00798940 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:16.335220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b421e411-a4d2-4153-bdbc-e87b00798940 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.163082 May 25 16:43:16.335998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1105/2231] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:16 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:16.588936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f9e9acd5-0070-4a5e-ac14-a4081e95217a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:16.589655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f9e9acd5-0070-4a5e-ac14-a4081e95217a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:16.597471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f9e9acd5-0070-4a5e-ac14-a4081e95217a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:16.598006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f9e9acd5-0070-4a5e-ac14-a4081e95217a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:16.598890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f9e9acd5-0070-4a5e-ac14-a4081e95217a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:16.599445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f9e9acd5-0070-4a5e-ac14-a4081e95217a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:16.686456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:16.687093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:16.693717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:16.694164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:16.694922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:16.695389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:16.745320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f9e9acd5-0070-4a5e-ac14-a4081e95217a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:16.753787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f9e9acd5-0070-4a5e-ac14-a4081e95217a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.171293 May 25 16:43:16.754417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1127/2232] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:16 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:16.760826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:16.772322 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:16.774549 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:16.775716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-eb64325e-d2a5-4f0b-8c77-45cb110a9086 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.093394 May 25 16:43:16.776268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1106/2233] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:16 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:16.975602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-adcc3ba0-afe1-4ba2-8ed2-742273166179 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:16.976452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-adcc3ba0-afe1-4ba2-8ed2-742273166179 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:16.983404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-adcc3ba0-afe1-4ba2-8ed2-742273166179 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:16.983863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-adcc3ba0-afe1-4ba2-8ed2-742273166179 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:16.984614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-adcc3ba0-afe1-4ba2-8ed2-742273166179 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:16.985056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-adcc3ba0-afe1-4ba2-8ed2-742273166179 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:17.049467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-adcc3ba0-afe1-4ba2-8ed2-742273166179 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:17.061408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-adcc3ba0-afe1-4ba2-8ed2-742273166179 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091381 May 25 16:43:17.062111 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1128/2234] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:16 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:17.355742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fe0c49f1-9da2-419d-8682-f1645c1abe99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:17.356321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fe0c49f1-9da2-419d-8682-f1645c1abe99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:17.366296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fe0c49f1-9da2-419d-8682-f1645c1abe99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:17.366906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fe0c49f1-9da2-419d-8682-f1645c1abe99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:17.368001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fe0c49f1-9da2-419d-8682-f1645c1abe99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:17.368432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fe0c49f1-9da2-419d-8682-f1645c1abe99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:17.522056 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fe0c49f1-9da2-419d-8682-f1645c1abe99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:17.530562 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fe0c49f1-9da2-419d-8682-f1645c1abe99 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.181091 May 25 16:43:17.531173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1107/2235] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:17 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:17.775664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5c23890a-5177-4ebc-96c7-fe25f16ca468 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:17.776723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5c23890a-5177-4ebc-96c7-fe25f16ca468 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:17.786988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c23890a-5177-4ebc-96c7-fe25f16ca468 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:17.787781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c23890a-5177-4ebc-96c7-fe25f16ca468 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:17.789373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c23890a-5177-4ebc-96c7-fe25f16ca468 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:17.789973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5c23890a-5177-4ebc-96c7-fe25f16ca468 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:17.794873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:17.795509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:17.801809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:17.802259 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:17.803014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:17.803495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:17.882557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:17.891699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:17.893234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:17.894374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-899b254c-bcbc-4edb-b853-39ed11d39547 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.103352 May 25 16:43:17.894934 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1108/2236] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:17 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:17.945422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5c23890a-5177-4ebc-96c7-fe25f16ca468 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:17.953905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5c23890a-5177-4ebc-96c7-fe25f16ca468 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.185007 May 25 16:43:17.954484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1129/2237] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:17 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:18.083456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-01582665-fcc7-4db2-9efe-e6b0bb9f0e0a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:18.084457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-01582665-fcc7-4db2-9efe-e6b0bb9f0e0a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:18.091728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-01582665-fcc7-4db2-9efe-e6b0bb9f0e0a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:18.092210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-01582665-fcc7-4db2-9efe-e6b0bb9f0e0a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:18.092963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-01582665-fcc7-4db2-9efe-e6b0bb9f0e0a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:18.093396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-01582665-fcc7-4db2-9efe-e6b0bb9f0e0a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:18.156120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-01582665-fcc7-4db2-9efe-e6b0bb9f0e0a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:18.164692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-01582665-fcc7-4db2-9efe-e6b0bb9f0e0a tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.087354 May 25 16:43:18.165253 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1109/2238] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:18 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:18.552439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-820d112b-f8ba-4520-b361-e158d85a20d7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:18.553458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-820d112b-f8ba-4520-b361-e158d85a20d7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:18.563779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-820d112b-f8ba-4520-b361-e158d85a20d7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:18.564533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-820d112b-f8ba-4520-b361-e158d85a20d7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:18.565792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-820d112b-f8ba-4520-b361-e158d85a20d7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:18.566523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-820d112b-f8ba-4520-b361-e158d85a20d7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:18.708695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-820d112b-f8ba-4520-b361-e158d85a20d7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:18.717409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-820d112b-f8ba-4520-b361-e158d85a20d7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.171610 May 25 16:43:18.717988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1130/2239] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:18 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:18.916167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:18.917013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:18.927978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:18.928741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:18.929998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:18.930612 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:18.976376 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-344193a9-a79a-405b-8db6-1fa1fb03f1cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:18.976991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-344193a9-a79a-405b-8db6-1fa1fb03f1cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:18.984133 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-344193a9-a79a-405b-8db6-1fa1fb03f1cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:18.984587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-344193a9-a79a-405b-8db6-1fa1fb03f1cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:18.985350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-344193a9-a79a-405b-8db6-1fa1fb03f1cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:18.985823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-344193a9-a79a-405b-8db6-1fa1fb03f1cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:19.003440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:19.009443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:19.010942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:19.012123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-17071f56-f2ec-4842-b633-924d1c22e1f2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.103115 May 25 16:43:19.012661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1110/2240] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:19.124511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-344193a9-a79a-405b-8db6-1fa1fb03f1cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:19.132666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-344193a9-a79a-405b-8db6-1fa1fb03f1cf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.160179 May 25 16:43:19.133324 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1131/2241] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:18 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:19.182256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f84cdff0-93fa-43a8-b211-34afce22cd6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:19.182809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f84cdff0-93fa-43a8-b211-34afce22cd6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:19.189301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f84cdff0-93fa-43a8-b211-34afce22cd6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:19.189782 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f84cdff0-93fa-43a8-b211-34afce22cd6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:19.190584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f84cdff0-93fa-43a8-b211-34afce22cd6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:19.191011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f84cdff0-93fa-43a8-b211-34afce22cd6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:19.252447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f84cdff0-93fa-43a8-b211-34afce22cd6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:19.260644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f84cdff0-93fa-43a8-b211-34afce22cd6e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.082430 May 25 16:43:19.261216 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1111/2242] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:19 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:19.738731 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9bd7b6bc-3335-44cf-a283-497d301e4b03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:19.739723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9bd7b6bc-3335-44cf-a283-497d301e4b03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:19.747410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9bd7b6bc-3335-44cf-a283-497d301e4b03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:19.747850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9bd7b6bc-3335-44cf-a283-497d301e4b03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:19.748597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9bd7b6bc-3335-44cf-a283-497d301e4b03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:19.749031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9bd7b6bc-3335-44cf-a283-497d301e4b03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:19.889638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9bd7b6bc-3335-44cf-a283-497d301e4b03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:19.902349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9bd7b6bc-3335-44cf-a283-497d301e4b03 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.169880 May 25 16:43:19.903213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1132/2243] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:19 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:20.034614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:20.035602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:20.044371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:20.044817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:20.045570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:20.045993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:20.060952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:20.109833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:20.118578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:20.120117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:20.121224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-461acebb-a736-4a02-afc8-8acbf10eb280 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.093076 May 25 16:43:20.121784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1112/2244] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:20 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:20.149990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-83ac17e9-9a90-4517-a4e0-4a43de17d867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:20.150606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-83ac17e9-9a90-4517-a4e0-4a43de17d867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:20.157543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83ac17e9-9a90-4517-a4e0-4a43de17d867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:20.157985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83ac17e9-9a90-4517-a4e0-4a43de17d867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:20.158742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83ac17e9-9a90-4517-a4e0-4a43de17d867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:20.159186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-83ac17e9-9a90-4517-a4e0-4a43de17d867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:20.277622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a500539c-3014-4530-b63b-138b008b4475 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:20.278326 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a500539c-3014-4530-b63b-138b008b4475 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:20.284564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a500539c-3014-4530-b63b-138b008b4475 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:20.285159 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a500539c-3014-4530-b63b-138b008b4475 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:20.285917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a500539c-3014-4530-b63b-138b008b4475 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:20.286380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a500539c-3014-4530-b63b-138b008b4475 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:20.293730 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-83ac17e9-9a90-4517-a4e0-4a43de17d867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:20.301815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-83ac17e9-9a90-4517-a4e0-4a43de17d867 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.155688 May 25 16:43:20.302385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1133/2245] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:20 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:20.348821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a500539c-3014-4530-b63b-138b008b4475 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:20.361797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a500539c-3014-4530-b63b-138b008b4475 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.087804 May 25 16:43:20.362713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1113/2246] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:20 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:20.923775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-44d56386-c893-4d9b-9720-093b0abbcdcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:20.924714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-44d56386-c893-4d9b-9720-093b0abbcdcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:20.935094 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44d56386-c893-4d9b-9720-093b0abbcdcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:20.935897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44d56386-c893-4d9b-9720-093b0abbcdcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:20.937148 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44d56386-c893-4d9b-9720-093b0abbcdcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:20.937887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-44d56386-c893-4d9b-9720-093b0abbcdcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:21.070684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-44d56386-c893-4d9b-9720-093b0abbcdcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:21.078979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-44d56386-c893-4d9b-9720-093b0abbcdcd tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.161714 May 25 16:43:21.079638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1134/2247] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:20 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:21.142040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:21.142903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:21.151487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:21.151932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:21.152696 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:21.153128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:21.223981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:21.232647 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:21.234139 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:21.235313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-88af33bb-7d6d-4d32-a949-8d0d6aa573e4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098803 May 25 16:43:21.235866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1114/2248] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:21 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:21.322933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f3ab3caa-6231-4b12-8567-6c5404cc18bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:21.323989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f3ab3caa-6231-4b12-8567-6c5404cc18bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:21.335004 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3ab3caa-6231-4b12-8567-6c5404cc18bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:21.335850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3ab3caa-6231-4b12-8567-6c5404cc18bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:21.336909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3ab3caa-6231-4b12-8567-6c5404cc18bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:21.337338 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f3ab3caa-6231-4b12-8567-6c5404cc18bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:21.380327 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0207a526-54d3-47db-86bf-692ba50bb01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:21.381013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0207a526-54d3-47db-86bf-692ba50bb01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:21.387686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0207a526-54d3-47db-86bf-692ba50bb01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:21.388433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0207a526-54d3-47db-86bf-692ba50bb01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:21.389306 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0207a526-54d3-47db-86bf-692ba50bb01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:21.389735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0207a526-54d3-47db-86bf-692ba50bb01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:21.463664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0207a526-54d3-47db-86bf-692ba50bb01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:21.471829 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0207a526-54d3-47db-86bf-692ba50bb01b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.095294 May 25 16:43:21.472300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1115/2249] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:21 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:21.494566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f3ab3caa-6231-4b12-8567-6c5404cc18bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:21.503005 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f3ab3caa-6231-4b12-8567-6c5404cc18bc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.186416 May 25 16:43:21.503631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1135/2250] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:21 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:22.097752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-66ac6eb6-4315-4a2d-881b-090b1156b480 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:22.098359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-66ac6eb6-4315-4a2d-881b-090b1156b480 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:22.104688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66ac6eb6-4315-4a2d-881b-090b1156b480 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:22.105158 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66ac6eb6-4315-4a2d-881b-090b1156b480 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:22.105925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66ac6eb6-4315-4a2d-881b-090b1156b480 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:22.106343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-66ac6eb6-4315-4a2d-881b-090b1156b480 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:22.239580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-66ac6eb6-4315-4a2d-881b-090b1156b480 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:22.252225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-66ac6eb6-4315-4a2d-881b-090b1156b480 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.158084 May 25 16:43:22.252891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1116/2251] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:22 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:22.256167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:22.256867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:22.265912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:22.265912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:22.266914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:22.266914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:22.328914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:22.338157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:22.339706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:22.340840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-20f7dd9b-bb2b-4fee-a67c-684a9e515210 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.090858 May 25 16:43:22.341403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1136/2252] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:22 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:22.489616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4c38d5c5-3384-4df6-8e99-5490a1613aa9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:22.490256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4c38d5c5-3384-4df6-8e99-5490a1613aa9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:22.497062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c38d5c5-3384-4df6-8e99-5490a1613aa9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:22.497536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c38d5c5-3384-4df6-8e99-5490a1613aa9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:22.498299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c38d5c5-3384-4df6-8e99-5490a1613aa9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:22.498731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4c38d5c5-3384-4df6-8e99-5490a1613aa9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:22.520369 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c6bf2c2c-1490-44c9-9b10-83a2a1874e80 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:22.520985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c6bf2c2c-1490-44c9-9b10-83a2a1874e80 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:22.527387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6bf2c2c-1490-44c9-9b10-83a2a1874e80 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:22.527872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6bf2c2c-1490-44c9-9b10-83a2a1874e80 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:22.528626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6bf2c2c-1490-44c9-9b10-83a2a1874e80 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:22.529061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6bf2c2c-1490-44c9-9b10-83a2a1874e80 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:22.567060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4c38d5c5-3384-4df6-8e99-5490a1613aa9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:22.575544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4c38d5c5-3384-4df6-8e99-5490a1613aa9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090347 May 25 16:43:22.576162 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1117/2253] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:22 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:22.664648 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c6bf2c2c-1490-44c9-9b10-83a2a1874e80 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:22.677401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c6bf2c2c-1490-44c9-9b10-83a2a1874e80 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.160547 May 25 16:43:22.678216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1137/2254] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:22 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:23.275749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-64b6b038-bbc0-4ac6-a61f-64df347ea810 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:23.276768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-64b6b038-bbc0-4ac6-a61f-64df347ea810 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:23.287944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64b6b038-bbc0-4ac6-a61f-64df347ea810 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:23.288693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64b6b038-bbc0-4ac6-a61f-64df347ea810 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:23.290276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64b6b038-bbc0-4ac6-a61f-64df347ea810 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:23.290995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-64b6b038-bbc0-4ac6-a61f-64df347ea810 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:23.358611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:23.359187 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:23.366043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:23.366501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:23.367291 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:23.367770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:23.385640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:23.453480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:23.457627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-64b6b038-bbc0-4ac6-a61f-64df347ea810 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:23.463092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:23.464635 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:23.465794 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f58ebaf7-42f7-406e-98c6-12ac4f27e6d2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.111240 May 25 16:43:23.466160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-64b6b038-bbc0-4ac6-a61f-64df347ea810 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.197119 May 25 16:43:23.466681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1138/2255] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:23 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 25 16:43:23.466846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1118/2256] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:23 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:23.596813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3ffd304f-2380-45c6-965e-6d812f97b693 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:23.597826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3ffd304f-2380-45c6-965e-6d812f97b693 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:23.608150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ffd304f-2380-45c6-965e-6d812f97b693 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:23.609001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ffd304f-2380-45c6-965e-6d812f97b693 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:23.610150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ffd304f-2380-45c6-965e-6d812f97b693 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:23.610874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3ffd304f-2380-45c6-965e-6d812f97b693 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:23.678464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3ffd304f-2380-45c6-965e-6d812f97b693 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:23.686718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3ffd304f-2380-45c6-965e-6d812f97b693 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.096493 May 25 16:43:23.687301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1139/2257] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:23 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:23.695505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1073a446-9f28-4598-a2c0-3308196ac2ce tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:23.696162 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1073a446-9f28-4598-a2c0-3308196ac2ce tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:23.702318 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1073a446-9f28-4598-a2c0-3308196ac2ce tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:23.702766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1073a446-9f28-4598-a2c0-3308196ac2ce tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:23.703550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1073a446-9f28-4598-a2c0-3308196ac2ce tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:23.703985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1073a446-9f28-4598-a2c0-3308196ac2ce tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:23.844951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1073a446-9f28-4598-a2c0-3308196ac2ce tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:23.853759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1073a446-9f28-4598-a2c0-3308196ac2ce tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162062 May 25 16:43:23.854299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1119/2258] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:23 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:24.487474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:24.488305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c6942daa-7ac4-4b0e-98d9-4279a2c64062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:24.488984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:24.489161 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c6942daa-7ac4-4b0e-98d9-4279a2c64062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:24.500055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6942daa-7ac4-4b0e-98d9-4279a2c64062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:24.500825 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:24.501195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6942daa-7ac4-4b0e-98d9-4279a2c64062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:24.502255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:24.502527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6942daa-7ac4-4b0e-98d9-4279a2c64062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:24.503234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c6942daa-7ac4-4b0e-98d9-4279a2c64062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:24.504970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:24.505786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:24.595337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:24.604107 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:24.605618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:24.606770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5653cfec-062b-475f-a240-f1d1c85b1ff0 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.125840 May 25 16:43:24.607536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1140/2259] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:24 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:24.659497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c6942daa-7ac4-4b0e-98d9-4279a2c64062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:24.671177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c6942daa-7ac4-4b0e-98d9-4279a2c64062 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.189863 May 25 16:43:24.671995 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1120/2260] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:24 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:24.704642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a0af9804-1cb4-46d6-b089-a900d4d0a9a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:24.705273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a0af9804-1cb4-46d6-b089-a900d4d0a9a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:24.712065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a0af9804-1cb4-46d6-b089-a900d4d0a9a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:24.712513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a0af9804-1cb4-46d6-b089-a900d4d0a9a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:24.713265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a0af9804-1cb4-46d6-b089-a900d4d0a9a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:24.713699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a0af9804-1cb4-46d6-b089-a900d4d0a9a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:24.785471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a0af9804-1cb4-46d6-b089-a900d4d0a9a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:24.794215 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a0af9804-1cb4-46d6-b089-a900d4d0a9a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093443 May 25 16:43:24.794787 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1141/2261] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:24 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:24.875203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a98c8864-534b-47bc-9acf-7aa1c1ce3343 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:24.876193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a98c8864-534b-47bc-9acf-7aa1c1ce3343 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:24.886459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a98c8864-534b-47bc-9acf-7aa1c1ce3343 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:24.887221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a98c8864-534b-47bc-9acf-7aa1c1ce3343 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:24.888526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a98c8864-534b-47bc-9acf-7aa1c1ce3343 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:24.889242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a98c8864-534b-47bc-9acf-7aa1c1ce3343 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:25.022856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a98c8864-534b-47bc-9acf-7aa1c1ce3343 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:25.031349 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a98c8864-534b-47bc-9acf-7aa1c1ce3343 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162528 May 25 16:43:25.031888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1121/2262] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:24 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:25.628792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:25.629881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:25.640781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:25.641535 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:25.642796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:25.643581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:25.693809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6067cbe2-f0a3-432d-8295-832cd938751c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:25.694383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6067cbe2-f0a3-432d-8295-832cd938751c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:25.700985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6067cbe2-f0a3-432d-8295-832cd938751c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:25.701437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6067cbe2-f0a3-432d-8295-832cd938751c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:25.702198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6067cbe2-f0a3-432d-8295-832cd938751c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:25.702633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6067cbe2-f0a3-432d-8295-832cd938751c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:25.714824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:25.723741 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:25.725335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:25.726487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2f48e7a5-2a10-42fe-8eb2-1829d5c205b7 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.104298 May 25 16:43:25.727084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1142/2263] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:25 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:25.811652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-26213f9b-9e80-4a2a-92c9-382d5194a8eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:25.812260 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-26213f9b-9e80-4a2a-92c9-382d5194a8eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:25.818370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26213f9b-9e80-4a2a-92c9-382d5194a8eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:25.818816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26213f9b-9e80-4a2a-92c9-382d5194a8eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:25.819596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26213f9b-9e80-4a2a-92c9-382d5194a8eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:25.820030 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-26213f9b-9e80-4a2a-92c9-382d5194a8eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:25.841341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6067cbe2-f0a3-432d-8295-832cd938751c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:25.849410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6067cbe2-f0a3-432d-8295-832cd938751c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.159486 May 25 16:43:25.849976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1122/2264] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:25 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:25.882742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-26213f9b-9e80-4a2a-92c9-382d5194a8eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:25.894146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-26213f9b-9e80-4a2a-92c9-382d5194a8eb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.086119 May 25 16:43:25.895012 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1143/2265] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:25 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:26.053022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-87556fcc-91ac-452d-9d3b-f080f141b21d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:26.054026 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-87556fcc-91ac-452d-9d3b-f080f141b21d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:26.064399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87556fcc-91ac-452d-9d3b-f080f141b21d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:26.065151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87556fcc-91ac-452d-9d3b-f080f141b21d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:26.066393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87556fcc-91ac-452d-9d3b-f080f141b21d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:26.067116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-87556fcc-91ac-452d-9d3b-f080f141b21d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:26.207649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-87556fcc-91ac-452d-9d3b-f080f141b21d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:26.215974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-87556fcc-91ac-452d-9d3b-f080f141b21d tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.169529 May 25 16:43:26.216547 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1123/2266] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:26 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:26.749379 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:26.750353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:26.760340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:26.760793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:26.761544 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:26.761990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:26.831712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:26.840893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:26.842399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:26.843585 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-58c8b44b-1d34-44be-97b1-cb804fdee667 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.100856 May 25 16:43:26.844145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1144/2267] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:26.869939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e682098d-6144-49a5-9408-64c68a2c7814 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:26.870898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e682098d-6144-49a5-9408-64c68a2c7814 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:26.881573 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e682098d-6144-49a5-9408-64c68a2c7814 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:26.882314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e682098d-6144-49a5-9408-64c68a2c7814 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:26.883598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e682098d-6144-49a5-9408-64c68a2c7814 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:26.884065 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e682098d-6144-49a5-9408-64c68a2c7814 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:26.911981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9040237e-4a52-4424-bd4a-12992fff6806 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:26.912598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9040237e-4a52-4424-bd4a-12992fff6806 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:26.919558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9040237e-4a52-4424-bd4a-12992fff6806 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:26.920012 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9040237e-4a52-4424-bd4a-12992fff6806 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:26.920776 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9040237e-4a52-4424-bd4a-12992fff6806 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:26.921213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9040237e-4a52-4424-bd4a-12992fff6806 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:26.995066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9040237e-4a52-4424-bd4a-12992fff6806 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:27.003863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9040237e-4a52-4424-bd4a-12992fff6806 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.096303 May 25 16:43:27.014073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1145/2268] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:27.036002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e682098d-6144-49a5-9408-64c68a2c7814 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:27.044670 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e682098d-6144-49a5-9408-64c68a2c7814 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.181402 May 25 16:43:27.045224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1124/2269] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:26 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:27.241378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e7afbe21-b57a-4194-8d28-0be53e940d09 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:27.242491 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e7afbe21-b57a-4194-8d28-0be53e940d09 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:27.249582 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e7afbe21-b57a-4194-8d28-0be53e940d09 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:27.250079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e7afbe21-b57a-4194-8d28-0be53e940d09 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:27.250911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e7afbe21-b57a-4194-8d28-0be53e940d09 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:27.251440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e7afbe21-b57a-4194-8d28-0be53e940d09 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:27.401710 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e7afbe21-b57a-4194-8d28-0be53e940d09 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:27.414773 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e7afbe21-b57a-4194-8d28-0be53e940d09 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.179343 May 25 16:43:27.415758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1146/2270] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:27 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:27.865901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:27.866885 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:27.873748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:27.874385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:27.874944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:27.875426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:27.942382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:27.951399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:27.952891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:27.954029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-796a0dae-b269-4414-8c32-a59ae8b82551 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.094726 May 25 16:43:27.954578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1125/2271] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:28.061848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0ae7a342-6cf0-4ff3-93af-86a8d3506146 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:28.062481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0ae7a342-6cf0-4ff3-93af-86a8d3506146 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:28.069325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ae7a342-6cf0-4ff3-93af-86a8d3506146 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:28.069325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ae7a342-6cf0-4ff3-93af-86a8d3506146 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:28.070007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ae7a342-6cf0-4ff3-93af-86a8d3506146 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:28.070459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ae7a342-6cf0-4ff3-93af-86a8d3506146 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:28.146560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5a4d9781-08b7-4efb-8d89-dadbeb9b2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:28.147171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5a4d9781-08b7-4efb-8d89-dadbeb9b2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:28.153301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5a4d9781-08b7-4efb-8d89-dadbeb9b2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:28.153749 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5a4d9781-08b7-4efb-8d89-dadbeb9b2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:28.154510 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5a4d9781-08b7-4efb-8d89-dadbeb9b2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:28.154942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5a4d9781-08b7-4efb-8d89-dadbeb9b2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:28.226984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-5a4d9781-08b7-4efb-8d89-dadbeb9b2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:28.230265 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ae7a342-6cf0-4ff3-93af-86a8d3506146 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:28.235300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5a4d9781-08b7-4efb-8d89-dadbeb9b2413 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.215051 May 25 16:43:28.235888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1147/2272] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:28.239416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0ae7a342-6cf0-4ff3-93af-86a8d3506146 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.180456 May 25 16:43:28.239416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1126/2273] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:28 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:28.433290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-18637d7a-9520-4cf6-91c0-2dfd2ec76dc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:28.433909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-18637d7a-9520-4cf6-91c0-2dfd2ec76dc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:28.440033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-18637d7a-9520-4cf6-91c0-2dfd2ec76dc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:28.440489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-18637d7a-9520-4cf6-91c0-2dfd2ec76dc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:28.441251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-18637d7a-9520-4cf6-91c0-2dfd2ec76dc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:28.441688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-18637d7a-9520-4cf6-91c0-2dfd2ec76dc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:28.583400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-18637d7a-9520-4cf6-91c0-2dfd2ec76dc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:28.591859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-18637d7a-9520-4cf6-91c0-2dfd2ec76dc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162654 May 25 16:43:28.592440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1148/2274] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:28 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:29.096787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:29.097398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:29.103584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:29.104030 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:29.104784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:29.105203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:29.179826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:29.188396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:29.189886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:29.191010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-21170c45-191c-483e-8897-5a392949667a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.221194 May 25 16:43:29.191598 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1127/2275] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:29.256482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2a330f4f-f258-4c63-b12e-4ee3f323b890 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:29.257361 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2a330f4f-f258-4c63-b12e-4ee3f323b890 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:29.258429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-47aad961-3cc2-4053-80a5-d65cefa3e07d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:29.259490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-47aad961-3cc2-4053-80a5-d65cefa3e07d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:29.269035 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a330f4f-f258-4c63-b12e-4ee3f323b890 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:29.269803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a330f4f-f258-4c63-b12e-4ee3f323b890 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:29.271084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a330f4f-f258-4c63-b12e-4ee3f323b890 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:29.271886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2a330f4f-f258-4c63-b12e-4ee3f323b890 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:29.274722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47aad961-3cc2-4053-80a5-d65cefa3e07d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:29.275198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47aad961-3cc2-4053-80a5-d65cefa3e07d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:29.275994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47aad961-3cc2-4053-80a5-d65cefa3e07d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:29.276393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-47aad961-3cc2-4053-80a5-d65cefa3e07d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:29.480692 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2a330f4f-f258-4c63-b12e-4ee3f323b890 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:29.488831 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2a330f4f-f258-4c63-b12e-4ee3f323b890 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.238768 May 25 16:43:29.489381 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1149/2276] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:29 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:29.547188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-47aad961-3cc2-4053-80a5-d65cefa3e07d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:29.555674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-47aad961-3cc2-4053-80a5-d65cefa3e07d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.303393 May 25 16:43:29.556252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1128/2277] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:29 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:29.612479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-76b3a8a1-7a0c-4e52-ab78-84eed8880770 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:29.613387 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-76b3a8a1-7a0c-4e52-ab78-84eed8880770 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:29.621390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-76b3a8a1-7a0c-4e52-ab78-84eed8880770 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:29.621842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-76b3a8a1-7a0c-4e52-ab78-84eed8880770 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:29.622613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-76b3a8a1-7a0c-4e52-ab78-84eed8880770 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:29.623051 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-76b3a8a1-7a0c-4e52-ab78-84eed8880770 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:29.765658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-76b3a8a1-7a0c-4e52-ab78-84eed8880770 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:29.773873 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-76b3a8a1-7a0c-4e52-ab78-84eed8880770 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.167709 May 25 16:43:29.774446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1150/2278] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:29 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:30.208905 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:30.209527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:30.216556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:30.217010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:30.217778 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:30.218190 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:30.220860 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:30.322126 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:30.331183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:30.332867 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:30.334016 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4d978221-f181-45c3-bf29-7a5ccffa085a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.129457 May 25 16:43:30.334585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1129/2279] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:30 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:30.509602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ad3a0dbb-44dd-49fc-8123-1b3130cb5a5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:30.510358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ad3a0dbb-44dd-49fc-8123-1b3130cb5a5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:30.518814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad3a0dbb-44dd-49fc-8123-1b3130cb5a5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:30.519421 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad3a0dbb-44dd-49fc-8123-1b3130cb5a5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:30.520354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad3a0dbb-44dd-49fc-8123-1b3130cb5a5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:30.520886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ad3a0dbb-44dd-49fc-8123-1b3130cb5a5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:30.572633 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9078274c-e4ad-4d1d-ad3c-24bcee674d52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:30.573157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9078274c-e4ad-4d1d-ad3c-24bcee674d52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:30.579601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9078274c-e4ad-4d1d-ad3c-24bcee674d52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:30.580058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9078274c-e4ad-4d1d-ad3c-24bcee674d52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:30.580820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9078274c-e4ad-4d1d-ad3c-24bcee674d52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:30.581258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9078274c-e4ad-4d1d-ad3c-24bcee674d52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:30.587332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ad3a0dbb-44dd-49fc-8123-1b3130cb5a5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:30.595431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ad3a0dbb-44dd-49fc-8123-1b3130cb5a5e tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092082 May 25 16:43:30.599996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1151/2280] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:30 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:30.712217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9078274c-e4ad-4d1d-ad3c-24bcee674d52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:30.724662 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9078274c-e4ad-4d1d-ad3c-24bcee674d52 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.155828 May 25 16:43:30.725487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1130/2281] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:30 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:30.791944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-58aa086e-c38d-4725-b875-64a5815b267e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:30.792547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-58aa086e-c38d-4725-b875-64a5815b267e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:30.798811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58aa086e-c38d-4725-b875-64a5815b267e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:30.799279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58aa086e-c38d-4725-b875-64a5815b267e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:30.800033 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58aa086e-c38d-4725-b875-64a5815b267e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:30.800458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58aa086e-c38d-4725-b875-64a5815b267e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:30.938482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-58aa086e-c38d-4725-b875-64a5815b267e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:30.951384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-58aa086e-c38d-4725-b875-64a5815b267e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.163493 May 25 16:43:30.952127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1152/2282] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:30 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:31.353826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:31.354442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:31.361364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:31.361809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:31.362555 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:31.362954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:31.426479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:31.442988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:31.445374 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:31.446580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4cbb11ad-a95a-4318-8573-e438d24a315a tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.097988 May 25 16:43:31.447141 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1131/2283] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:31 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:31.616834 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c168705d-aba4-4827-be73-71a0dd8f4b3b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:31.617684 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c168705d-aba4-4827-be73-71a0dd8f4b3b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:31.624755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c168705d-aba4-4827-be73-71a0dd8f4b3b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:31.625203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c168705d-aba4-4827-be73-71a0dd8f4b3b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:31.625962 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c168705d-aba4-4827-be73-71a0dd8f4b3b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:31.626395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c168705d-aba4-4827-be73-71a0dd8f4b3b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:31.692587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c168705d-aba4-4827-be73-71a0dd8f4b3b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:31.700764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c168705d-aba4-4827-be73-71a0dd8f4b3b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090796 May 25 16:43:31.701330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1153/2284] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:31 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:31.743419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-58c881c0-cb8a-468a-8994-507040ecec9f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:31.744037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-58c881c0-cb8a-468a-8994-507040ecec9f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:31.750483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58c881c0-cb8a-468a-8994-507040ecec9f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:31.750937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58c881c0-cb8a-468a-8994-507040ecec9f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:31.751743 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58c881c0-cb8a-468a-8994-507040ecec9f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:31.752178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58c881c0-cb8a-468a-8994-507040ecec9f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:31.892357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-58c881c0-cb8a-468a-8994-507040ecec9f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:31.905163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-58c881c0-cb8a-468a-8994-507040ecec9f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.165787 May 25 16:43:31.905985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1132/2285] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:31 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:31.971770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-09584bfe-329f-4e88-830e-0fa72187cf10 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:31.972878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-09584bfe-329f-4e88-830e-0fa72187cf10 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:31.980743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09584bfe-329f-4e88-830e-0fa72187cf10 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:31.981190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09584bfe-329f-4e88-830e-0fa72187cf10 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:31.981943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09584bfe-329f-4e88-830e-0fa72187cf10 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:31.982406 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09584bfe-329f-4e88-830e-0fa72187cf10 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:32.117884 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-09584bfe-329f-4e88-830e-0fa72187cf10 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:32.126155 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-09584bfe-329f-4e88-830e-0fa72187cf10 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.160898 May 25 16:43:32.126718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1154/2286] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:31 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:32.468575 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:32.469583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:32.480556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:32.482060 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:32.482821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:32.483363 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:32.552370 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:32.561063 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:32.562576 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:32.563736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6b3cdb70-49a1-4a93-8aa5-92795ec2c0a3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.101709 May 25 16:43:32.564296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1133/2287] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:32 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:32.721113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f5e32614-bb17-4df5-bffc-c56c468e0639 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:32.721981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f5e32614-bb17-4df5-bffc-c56c468e0639 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:32.731015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5e32614-bb17-4df5-bffc-c56c468e0639 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:32.731518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5e32614-bb17-4df5-bffc-c56c468e0639 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:32.732292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5e32614-bb17-4df5-bffc-c56c468e0639 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:32.732769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5e32614-bb17-4df5-bffc-c56c468e0639 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:32.735430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:32.802151 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f5e32614-bb17-4df5-bffc-c56c468e0639 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:32.814738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f5e32614-bb17-4df5-bffc-c56c468e0639 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.100138 May 25 16:43:32.815411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1155/2288] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:32 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:32.926735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c5e08844-1f78-4892-8ff6-11b58ef2aa45 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:32.927794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c5e08844-1f78-4892-8ff6-11b58ef2aa45 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:32.937803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c5e08844-1f78-4892-8ff6-11b58ef2aa45 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:32.938577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c5e08844-1f78-4892-8ff6-11b58ef2aa45 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:32.939866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c5e08844-1f78-4892-8ff6-11b58ef2aa45 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:32.940591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c5e08844-1f78-4892-8ff6-11b58ef2aa45 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:33.072229 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c5e08844-1f78-4892-8ff6-11b58ef2aa45 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:33.081624 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c5e08844-1f78-4892-8ff6-11b58ef2aa45 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.161313 May 25 16:43:33.082281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1134/2289] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:32 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:33.143544 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e1228c5f-18c4-4e17-9099-ea58a2951b73 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:33.144150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e1228c5f-18c4-4e17-9099-ea58a2951b73 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:33.150591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1228c5f-18c4-4e17-9099-ea58a2951b73 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:33.151059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1228c5f-18c4-4e17-9099-ea58a2951b73 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:33.151783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1228c5f-18c4-4e17-9099-ea58a2951b73 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:33.152211 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e1228c5f-18c4-4e17-9099-ea58a2951b73 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:33.292857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e1228c5f-18c4-4e17-9099-ea58a2951b73 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:33.305695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e1228c5f-18c4-4e17-9099-ea58a2951b73 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.165976 May 25 16:43:33.306263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1156/2290] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:33 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:33.590825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:33.591790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:33.602826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:33.603118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:33.603965 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:33.604400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:33.675264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:33.685272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:33.687008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:33.688184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-085e66b7-f782-48b2-9afd-f3d3532d5c3d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.104057 May 25 16:43:33.688723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1135/2291] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:33 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:33.837738 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ab8e90a9-aaa0-4777-b501-57327c3ef7dd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:33.838360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ab8e90a9-aaa0-4777-b501-57327c3ef7dd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:33.845917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ab8e90a9-aaa0-4777-b501-57327c3ef7dd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:33.846300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ab8e90a9-aaa0-4777-b501-57327c3ef7dd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:33.847074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ab8e90a9-aaa0-4777-b501-57327c3ef7dd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:33.847574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ab8e90a9-aaa0-4777-b501-57327c3ef7dd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:33.917728 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ab8e90a9-aaa0-4777-b501-57327c3ef7dd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:33.925805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ab8e90a9-aaa0-4777-b501-57327c3ef7dd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.094469 May 25 16:43:33.926349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1157/2292] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:33 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:34.099553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c4dc68a7-62e7-4826-80e4-fc09b4aa6bc8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:34.100390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c4dc68a7-62e7-4826-80e4-fc09b4aa6bc8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:34.107261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4dc68a7-62e7-4826-80e4-fc09b4aa6bc8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:34.107718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4dc68a7-62e7-4826-80e4-fc09b4aa6bc8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:34.108479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4dc68a7-62e7-4826-80e4-fc09b4aa6bc8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:34.108915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4dc68a7-62e7-4826-80e4-fc09b4aa6bc8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:34.260999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c4dc68a7-62e7-4826-80e4-fc09b4aa6bc8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:34.270076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c4dc68a7-62e7-4826-80e4-fc09b4aa6bc8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.174955 May 25 16:43:34.270622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1136/2293] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:34 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:34.324539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7788401f-1305-4029-a9cf-66d8a1d991ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:34.325149 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7788401f-1305-4029-a9cf-66d8a1d991ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:34.331847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7788401f-1305-4029-a9cf-66d8a1d991ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:34.332298 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7788401f-1305-4029-a9cf-66d8a1d991ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:34.333053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7788401f-1305-4029-a9cf-66d8a1d991ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:34.333487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7788401f-1305-4029-a9cf-66d8a1d991ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:34.471367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7788401f-1305-4029-a9cf-66d8a1d991ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:34.480159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7788401f-1305-4029-a9cf-66d8a1d991ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.159765 May 25 16:43:34.480716 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1158/2294] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:34 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:34.711261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:34.711919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:34.720814 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:34.721739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:34.722619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:34.723184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:34.790418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:34.799040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:34.800608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:34.802099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8a44e5a2-6acc-4412-ae46-38304fe72523 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098185 May 25 16:43:34.802648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1137/2295] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:34 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:34.945695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-244cc9f4-140e-4740-a5f2-71fa076535f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:34.946319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-244cc9f4-140e-4740-a5f2-71fa076535f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:34.953365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-244cc9f4-140e-4740-a5f2-71fa076535f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:34.953805 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-244cc9f4-140e-4740-a5f2-71fa076535f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:34.954557 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-244cc9f4-140e-4740-a5f2-71fa076535f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:34.954987 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-244cc9f4-140e-4740-a5f2-71fa076535f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:35.015939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-244cc9f4-140e-4740-a5f2-71fa076535f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:35.024207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-244cc9f4-140e-4740-a5f2-71fa076535f4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.082630 May 25 16:43:35.024767 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1159/2296] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:34 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:35.291748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-773c3af8-63af-4724-9700-3689c2b809b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:35.292753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-773c3af8-63af-4724-9700-3689c2b809b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:35.304177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-773c3af8-63af-4724-9700-3689c2b809b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:35.305014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-773c3af8-63af-4724-9700-3689c2b809b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:35.305802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-773c3af8-63af-4724-9700-3689c2b809b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:35.306234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-773c3af8-63af-4724-9700-3689c2b809b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:35.440742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-773c3af8-63af-4724-9700-3689c2b809b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:35.453168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-773c3af8-63af-4724-9700-3689c2b809b9 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.168159 May 25 16:43:35.453911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1138/2297] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:35 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:35.498153 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a73e9a5f-8233-44b8-bf2f-683956f2b102 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:35.499176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a73e9a5f-8233-44b8-bf2f-683956f2b102 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:35.505614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a73e9a5f-8233-44b8-bf2f-683956f2b102 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:35.506064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a73e9a5f-8233-44b8-bf2f-683956f2b102 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:35.506826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a73e9a5f-8233-44b8-bf2f-683956f2b102 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:35.507305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a73e9a5f-8233-44b8-bf2f-683956f2b102 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:35.642776 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a73e9a5f-8233-44b8-bf2f-683956f2b102 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:35.656401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a73e9a5f-8233-44b8-bf2f-683956f2b102 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.161818 May 25 16:43:35.657184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1160/2298] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:35 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:35.821000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:35.821613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:35.830343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:35.830787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:35.831563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:35.831990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:35.904776 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:35.917163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:35.919034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:35.920245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e3d48223-fb2f-47f8-8a99-deb8139f0d1f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.103401 May 25 16:43:35.920804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1139/2299] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:35 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:36.045294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f561aa98-0822-45d2-9d55-613778b316ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:36.046281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f561aa98-0822-45d2-9d55-613778b316ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:36.056676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f561aa98-0822-45d2-9d55-613778b316ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:36.057118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f561aa98-0822-45d2-9d55-613778b316ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:36.057870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f561aa98-0822-45d2-9d55-613778b316ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:36.058301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f561aa98-0822-45d2-9d55-613778b316ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:36.127951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f561aa98-0822-45d2-9d55-613778b316ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:36.140732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f561aa98-0822-45d2-9d55-613778b316ed tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.102098 May 25 16:43:36.141532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1161/2300] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:36 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:36.477543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b2898301-a075-4234-8210-be783b5cfd98 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:36.478399 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b2898301-a075-4234-8210-be783b5cfd98 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:36.485790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2898301-a075-4234-8210-be783b5cfd98 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:36.486320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2898301-a075-4234-8210-be783b5cfd98 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:36.487186 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2898301-a075-4234-8210-be783b5cfd98 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:36.487664 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b2898301-a075-4234-8210-be783b5cfd98 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:36.623539 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b2898301-a075-4234-8210-be783b5cfd98 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:36.632067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b2898301-a075-4234-8210-be783b5cfd98 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.161025 May 25 16:43:36.632645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1140/2301] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:36 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:36.676821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3639dd31-65d1-4f2c-abe5-b7c70ec570ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:36.677366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3639dd31-65d1-4f2c-abe5-b7c70ec570ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:36.683797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3639dd31-65d1-4f2c-abe5-b7c70ec570ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:36.684246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3639dd31-65d1-4f2c-abe5-b7c70ec570ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:36.684995 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3639dd31-65d1-4f2c-abe5-b7c70ec570ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:36.685424 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3639dd31-65d1-4f2c-abe5-b7c70ec570ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:36.822343 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3639dd31-65d1-4f2c-abe5-b7c70ec570ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:36.835057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3639dd31-65d1-4f2c-abe5-b7c70ec570ec tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.162169 May 25 16:43:36.835938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1162/2302] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:36 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:36.943999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:36.944536 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:36.951058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:36.951543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:36.952298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:36.952732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:37.018807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:37.027605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:37.029082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:37.030215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-be9ed899-3f3a-4124-989e-f6c9f420b10c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.090255 May 25 16:43:37.030745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1141/2303] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:36 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:37.164532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-99cce088-7dd8-4cb4-91fd-3ba33d3dbec3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:37.165098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-99cce088-7dd8-4cb4-91fd-3ba33d3dbec3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:37.172037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-99cce088-7dd8-4cb4-91fd-3ba33d3dbec3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:37.172536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-99cce088-7dd8-4cb4-91fd-3ba33d3dbec3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:37.173294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-99cce088-7dd8-4cb4-91fd-3ba33d3dbec3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:37.173696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-99cce088-7dd8-4cb4-91fd-3ba33d3dbec3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:37.236458 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-99cce088-7dd8-4cb4-91fd-3ba33d3dbec3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:37.244622 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-99cce088-7dd8-4cb4-91fd-3ba33d3dbec3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.087018 May 25 16:43:37.245202 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1163/2304] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:37 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:37.653741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-80253c04-657e-4bec-afa1-36156809a400 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:37.654745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-80253c04-657e-4bec-afa1-36156809a400 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:37.665512 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80253c04-657e-4bec-afa1-36156809a400 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:37.666230 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80253c04-657e-4bec-afa1-36156809a400 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:37.667549 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80253c04-657e-4bec-afa1-36156809a400 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:37.668288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80253c04-657e-4bec-afa1-36156809a400 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:37.813919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-80253c04-657e-4bec-afa1-36156809a400 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:37.822073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-80253c04-657e-4bec-afa1-36156809a400 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.174837 May 25 16:43:37.822639 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1142/2305] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:37 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:37.855050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-59fe0777-a7f8-430a-97c5-0a1e1ca8bfb3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:37.855985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-59fe0777-a7f8-430a-97c5-0a1e1ca8bfb3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:37.864903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59fe0777-a7f8-430a-97c5-0a1e1ca8bfb3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:37.865545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59fe0777-a7f8-430a-97c5-0a1e1ca8bfb3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:37.866715 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59fe0777-a7f8-430a-97c5-0a1e1ca8bfb3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:37.867472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59fe0777-a7f8-430a-97c5-0a1e1ca8bfb3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:38.005711 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-59fe0777-a7f8-430a-97c5-0a1e1ca8bfb3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:38.023720 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-59fe0777-a7f8-430a-97c5-0a1e1ca8bfb3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.174315 May 25 16:43:38.024533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1164/2306] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:37 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:38.049145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:38.050067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:38.058071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:38.058675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:38.059916 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:38.060563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:38.137044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:38.145963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:38.147483 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:38.148711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4da8c891-3698-4c74-9b22-c5c5f7840df9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.103778 May 25 16:43:38.149288 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1143/2307] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:38 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:38.265011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4c94a24e-beba-4db7-8b98-f33a432483aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:38.265885 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4c94a24e-beba-4db7-8b98-f33a432483aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:38.275774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c94a24e-beba-4db7-8b98-f33a432483aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:38.276226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c94a24e-beba-4db7-8b98-f33a432483aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:38.276980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c94a24e-beba-4db7-8b98-f33a432483aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:38.277414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4c94a24e-beba-4db7-8b98-f33a432483aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:38.347919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4c94a24e-beba-4db7-8b98-f33a432483aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:38.356301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4c94a24e-beba-4db7-8b98-f33a432483aa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.097847 May 25 16:43:38.356869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1165/2308] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:38 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:38.843400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c7432ad7-72d0-4d16-8bfc-bbf3731be950 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:38.844390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c7432ad7-72d0-4d16-8bfc-bbf3731be950 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:38.853589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7432ad7-72d0-4d16-8bfc-bbf3731be950 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:38.854039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7432ad7-72d0-4d16-8bfc-bbf3731be950 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:38.854787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7432ad7-72d0-4d16-8bfc-bbf3731be950 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:38.855258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c7432ad7-72d0-4d16-8bfc-bbf3731be950 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:38.989774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c7432ad7-72d0-4d16-8bfc-bbf3731be950 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:38.998784 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c7432ad7-72d0-4d16-8bfc-bbf3731be950 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.161943 May 25 16:43:38.999383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1144/2309] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:38 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:39.047484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ea4cda9d-ec73-496d-9964-c0521bb6dde7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:39.048090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ea4cda9d-ec73-496d-9964-c0521bb6dde7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:39.054911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea4cda9d-ec73-496d-9964-c0521bb6dde7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:39.055416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea4cda9d-ec73-496d-9964-c0521bb6dde7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:39.056174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea4cda9d-ec73-496d-9964-c0521bb6dde7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:39.056603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea4cda9d-ec73-496d-9964-c0521bb6dde7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:39.167787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:39.168400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:39.170668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:39.176285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:39.176737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:39.177502 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:39.177936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:39.193413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ea4cda9d-ec73-496d-9964-c0521bb6dde7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:39.201492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ea4cda9d-ec73-496d-9964-c0521bb6dde7 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.157830 May 25 16:43:39.202059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1166/2310] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:39 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:39.242793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:39.245070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:39.252933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:39.254447 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:39.255619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-096d586c-1bf5-40e8-bdef-2f522fe2da44 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.092224 May 25 16:43:39.256182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1145/2311] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:39 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:39.377278 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8596a279-a4e3-4c07-8663-c1c15b99d628 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:39.378273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8596a279-a4e3-4c07-8663-c1c15b99d628 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:39.387349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8596a279-a4e3-4c07-8663-c1c15b99d628 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:39.387800 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8596a279-a4e3-4c07-8663-c1c15b99d628 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:39.388618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8596a279-a4e3-4c07-8663-c1c15b99d628 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:39.388985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-8596a279-a4e3-4c07-8663-c1c15b99d628 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:39.453436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8596a279-a4e3-4c07-8663-c1c15b99d628 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:39.461621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8596a279-a4e3-4c07-8663-c1c15b99d628 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090750 May 25 16:43:39.462191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1167/2312] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:39 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:40.020423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e00ecb7a-b4d8-4349-b2f4-0ad73fef8712 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:40.021274 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e00ecb7a-b4d8-4349-b2f4-0ad73fef8712 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:40.028726 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e00ecb7a-b4d8-4349-b2f4-0ad73fef8712 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:40.029178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e00ecb7a-b4d8-4349-b2f4-0ad73fef8712 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:40.029933 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e00ecb7a-b4d8-4349-b2f4-0ad73fef8712 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:40.030365 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e00ecb7a-b4d8-4349-b2f4-0ad73fef8712 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:40.164165 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e00ecb7a-b4d8-4349-b2f4-0ad73fef8712 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:40.173293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e00ecb7a-b4d8-4349-b2f4-0ad73fef8712 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.159545 May 25 16:43:40.173889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1146/2313] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:40 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:40.218904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d128dfc9-d9c2-42d3-bb62-65960ce890c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:40.219558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d128dfc9-d9c2-42d3-bb62-65960ce890c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:40.225906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d128dfc9-d9c2-42d3-bb62-65960ce890c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:40.226365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d128dfc9-d9c2-42d3-bb62-65960ce890c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:40.227197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d128dfc9-d9c2-42d3-bb62-65960ce890c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:40.227618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d128dfc9-d9c2-42d3-bb62-65960ce890c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:40.273956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:40.274556 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:40.280983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:40.281430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:40.282191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:40.282615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:40.356229 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:40.364877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:40.366373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:40.367570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e19393ec-b130-47d9-85a7-276e9ae7c329 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.097518 May 25 16:43:40.368098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1147/2314] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:40 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:40.386312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d128dfc9-d9c2-42d3-bb62-65960ce890c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:40.394494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d128dfc9-d9c2-42d3-bb62-65960ce890c8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.179524 May 25 16:43:40.395059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1168/2315] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:40 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:40.481563 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a10f3f1b-3aed-4196-ac44-b7f511cc9d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:40.482686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a10f3f1b-3aed-4196-ac44-b7f511cc9d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:40.492798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a10f3f1b-3aed-4196-ac44-b7f511cc9d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:40.493245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a10f3f1b-3aed-4196-ac44-b7f511cc9d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:40.494003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a10f3f1b-3aed-4196-ac44-b7f511cc9d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:40.494435 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a10f3f1b-3aed-4196-ac44-b7f511cc9d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:40.558559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a10f3f1b-3aed-4196-ac44-b7f511cc9d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:40.567264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a10f3f1b-3aed-4196-ac44-b7f511cc9d83 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092201 May 25 16:43:40.567816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1148/2316] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:40 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:41.194745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0eafc21f-34bf-4d18-bacb-a8792c68af2f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:41.195697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0eafc21f-34bf-4d18-bacb-a8792c68af2f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:41.202050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0eafc21f-34bf-4d18-bacb-a8792c68af2f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:41.202505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0eafc21f-34bf-4d18-bacb-a8792c68af2f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:41.203305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0eafc21f-34bf-4d18-bacb-a8792c68af2f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:41.203744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0eafc21f-34bf-4d18-bacb-a8792c68af2f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:41.336127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0eafc21f-34bf-4d18-bacb-a8792c68af2f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:41.344461 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0eafc21f-34bf-4d18-bacb-a8792c68af2f tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.156236 May 25 16:43:41.345036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1169/2317] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:41 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:41.386648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:41.387229 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:41.394115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:41.394565 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:41.395346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:41.395761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:41.413162 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0ace2365-92e9-464f-9bba-bdcb2ea3a869 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:41.413789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0ace2365-92e9-464f-9bba-bdcb2ea3a869 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:41.421046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0ace2365-92e9-464f-9bba-bdcb2ea3a869 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:41.421501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0ace2365-92e9-464f-9bba-bdcb2ea3a869 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:41.422246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0ace2365-92e9-464f-9bba-bdcb2ea3a869 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:41.422678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0ace2365-92e9-464f-9bba-bdcb2ea3a869 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:41.467964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:41.476844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:41.478327 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:41.479487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-807b22d6-255d-48a0-bc9f-ce8e4c54c8ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.097089 May 25 16:43:41.480040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1149/2318] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:41 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:41.568034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0ace2365-92e9-464f-9bba-bdcb2ea3a869 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:41.581961 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0ace2365-92e9-464f-9bba-bdcb2ea3a869 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.172387 May 25 16:43:41.583389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1170/2319] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:41 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:41.589610 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d9b914e2-48ad-4c07-a9ae-054a6a85d4da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:41.590221 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d9b914e2-48ad-4c07-a9ae-054a6a85d4da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:41.596560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d9b914e2-48ad-4c07-a9ae-054a6a85d4da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:41.597007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d9b914e2-48ad-4c07-a9ae-054a6a85d4da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:41.597754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d9b914e2-48ad-4c07-a9ae-054a6a85d4da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:41.598176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d9b914e2-48ad-4c07-a9ae-054a6a85d4da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:41.659587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d9b914e2-48ad-4c07-a9ae-054a6a85d4da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:41.668442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d9b914e2-48ad-4c07-a9ae-054a6a85d4da tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.082825 May 25 16:43:41.669013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1150/2320] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:41 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:42.366161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e90fb023-70ff-4133-9114-22a959858ba0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:42.367239 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e90fb023-70ff-4133-9114-22a959858ba0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:42.373768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e90fb023-70ff-4133-9114-22a959858ba0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:42.374218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e90fb023-70ff-4133-9114-22a959858ba0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:42.374972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e90fb023-70ff-4133-9114-22a959858ba0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:42.375445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e90fb023-70ff-4133-9114-22a959858ba0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:42.500489 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:42.501107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:42.510233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:42.510717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:42.511509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:42.511938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:42.783764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:42.795418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:42.797284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:42.798487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-35ab9999-0dc1-4ea6-9cdd-48380b53e248 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.304332 May 25 16:43:42.799051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1151/2321] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:42 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:42.805590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e90fb023-70ff-4133-9114-22a959858ba0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:42.808301 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:42.808777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ea99d38f-7404-4597-bda0-ec37ec94f83f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:42.809479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ea99d38f-7404-4597-bda0-ec37ec94f83f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:42.817020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea99d38f-7404-4597-bda0-ec37ec94f83f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:42.817653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea99d38f-7404-4597-bda0-ec37ec94f83f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:42.818472 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea99d38f-7404-4597-bda0-ec37ec94f83f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:42.818637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e90fb023-70ff-4133-9114-22a959858ba0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.458709 May 25 16:43:42.818913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ea99d38f-7404-4597-bda0-ec37ec94f83f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:42.819652 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1171/2322] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:42 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:42.825561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-812c3bfc-80b3-4274-bf8d-98d416cd4338 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:42.826170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-812c3bfc-80b3-4274-bf8d-98d416cd4338 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:42.832215 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-812c3bfc-80b3-4274-bf8d-98d416cd4338 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:42.832645 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-812c3bfc-80b3-4274-bf8d-98d416cd4338 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:42.833411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-812c3bfc-80b3-4274-bf8d-98d416cd4338 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:42.833833 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-812c3bfc-80b3-4274-bf8d-98d416cd4338 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:42.910184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-812c3bfc-80b3-4274-bf8d-98d416cd4338 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:42.918925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-812c3bfc-80b3-4274-bf8d-98d416cd4338 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.097739 May 25 16:43:42.919548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1172/2323] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:42 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:42.977485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ea99d38f-7404-4597-bda0-ec37ec94f83f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:42.987902 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ea99d38f-7404-4597-bda0-ec37ec94f83f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.187881 May 25 16:43:42.988484 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1152/2324] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:42 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:43.824200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:43.825245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:43.836223 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:43.836693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:43.837489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:43.837937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:43.839006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5f3dca3b-b675-4413-abaa-1e668a6c8dae tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:43.839671 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5f3dca3b-b675-4413-abaa-1e668a6c8dae tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:43.846850 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f3dca3b-b675-4413-abaa-1e668a6c8dae tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:43.847353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f3dca3b-b675-4413-abaa-1e668a6c8dae tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:43.848149 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f3dca3b-b675-4413-abaa-1e668a6c8dae tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:43.848603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5f3dca3b-b675-4413-abaa-1e668a6c8dae tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:43.907518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:43.916771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:43.918893 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:43.920625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3e4e2502-a518-443a-9419-d1b3220b1b47 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.102650 May 25 16:43:43.921563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1173/2325] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:43 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:43.937840 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e802087a-abbe-459b-9c2d-0a63061292a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:43.938599 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e802087a-abbe-459b-9c2d-0a63061292a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:43.944761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e802087a-abbe-459b-9c2d-0a63061292a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:43.945326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e802087a-abbe-459b-9c2d-0a63061292a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:43.946086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e802087a-abbe-459b-9c2d-0a63061292a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:43.946509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e802087a-abbe-459b-9c2d-0a63061292a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:44.001272 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5f3dca3b-b675-4413-abaa-1e668a6c8dae tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:44.009800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5f3dca3b-b675-4413-abaa-1e668a6c8dae tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.175887 May 25 16:43:44.010371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1153/2326] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:43 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:44.014767 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ad6b6277-dc4f-4be2-96d2-1b39f1702e37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:44.015392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ad6b6277-dc4f-4be2-96d2-1b39f1702e37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:44.016992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e802087a-abbe-459b-9c2d-0a63061292a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:44.021400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad6b6277-dc4f-4be2-96d2-1b39f1702e37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:44.021845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad6b6277-dc4f-4be2-96d2-1b39f1702e37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:44.022599 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad6b6277-dc4f-4be2-96d2-1b39f1702e37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:44.023029 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad6b6277-dc4f-4be2-96d2-1b39f1702e37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:44.025407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e802087a-abbe-459b-9c2d-0a63061292a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091402 May 25 16:43:44.025974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1174/2327] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:43 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:44.160529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ad6b6277-dc4f-4be2-96d2-1b39f1702e37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:44.169168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ad6b6277-dc4f-4be2-96d2-1b39f1702e37 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.157831 May 25 16:43:44.169758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1154/2328] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:44 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:44.944180 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:44.945426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:44.952462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:44.952909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:44.953665 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:44.954096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:45.016398 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:45.025314 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:45.026813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:45.027517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-40e67397-5bdd-4c11-88ea-face51c0f457 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:45.028001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-98fcac31-e85a-4fd4-879f-2e826b39df4d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.090396 May 25 16:43:45.028112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-40e67397-5bdd-4c11-88ea-face51c0f457 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:45.028538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1175/2329] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:44 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:45.035212 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-40e67397-5bdd-4c11-88ea-face51c0f457 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:45.035700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-40e67397-5bdd-4c11-88ea-face51c0f457 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:45.036437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-40e67397-5bdd-4c11-88ea-face51c0f457 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:45.036891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-40e67397-5bdd-4c11-88ea-face51c0f457 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:45.044860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-960da944-55ed-404e-996f-05c64184c7fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:45.045485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-960da944-55ed-404e-996f-05c64184c7fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:45.047297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:45.052668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-960da944-55ed-404e-996f-05c64184c7fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:45.053127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-960da944-55ed-404e-996f-05c64184c7fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:45.053888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-960da944-55ed-404e-996f-05c64184c7fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:45.054323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-960da944-55ed-404e-996f-05c64184c7fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:45.134645 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-960da944-55ed-404e-996f-05c64184c7fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:45.142534 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-960da944-55ed-404e-996f-05c64184c7fa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.101361 May 25 16:43:45.143115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1176/2330] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:45 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:43:45.185946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f03a9d43-899e-478f-a78c-0351eb2e94cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:45.186543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f03a9d43-899e-478f-a78c-0351eb2e94cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:45.192685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f03a9d43-899e-478f-a78c-0351eb2e94cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:45.193139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f03a9d43-899e-478f-a78c-0351eb2e94cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:45.194025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f03a9d43-899e-478f-a78c-0351eb2e94cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:45.194350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f03a9d43-899e-478f-a78c-0351eb2e94cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:45.196637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-40e67397-5bdd-4c11-88ea-face51c0f457 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:45.205035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-40e67397-5bdd-4c11-88ea-face51c0f457 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.181449 May 25 16:43:45.205806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1155/2331] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:45 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:45.338996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f03a9d43-899e-478f-a78c-0351eb2e94cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:45.351519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f03a9d43-899e-478f-a78c-0351eb2e94cd tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.169155 May 25 16:43:45.352277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1177/2332] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:45 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:46.046340 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:46.046914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:46.053439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:46.053970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:46.054886 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:46.055661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:46.120781 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:46.129477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:46.130983 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:46.132174 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-49abc48e-d4c8-4a11-b836-4dff973c2f95 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.090026 May 25 16:43:46.132799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1156/2333] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:46 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:46.159856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-92a22243-3343-4bec-87f2-4c1e2ed821e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:46.160475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-92a22243-3343-4bec-87f2-4c1e2ed821e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:46.167662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-92a22243-3343-4bec-87f2-4c1e2ed821e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:46.168116 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-92a22243-3343-4bec-87f2-4c1e2ed821e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:46.168875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-92a22243-3343-4bec-87f2-4c1e2ed821e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:46.169312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-92a22243-3343-4bec-87f2-4c1e2ed821e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:46.223383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0ffa344d-e892-43c0-bef5-af778cb0145d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:46.223984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0ffa344d-e892-43c0-bef5-af778cb0145d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:46.230604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ffa344d-e892-43c0-bef5-af778cb0145d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:46.231022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ffa344d-e892-43c0-bef5-af778cb0145d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:46.231844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ffa344d-e892-43c0-bef5-af778cb0145d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:46.232267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0ffa344d-e892-43c0-bef5-af778cb0145d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:46.241037 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-92a22243-3343-4bec-87f2-4c1e2ed821e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:46.249297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-92a22243-3343-4bec-87f2-4c1e2ed821e6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093284 May 25 16:43:46.249882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1178/2334] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:46 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:46.362635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0ffa344d-e892-43c0-bef5-af778cb0145d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:46.371418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0ffa344d-e892-43c0-bef5-af778cb0145d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.152115 May 25 16:43:46.372036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1157/2335] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:46 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:46.374416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ee2eeff3-db81-4da1-9361-55fae10d4310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:46.375124 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ee2eeff3-db81-4da1-9361-55fae10d4310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:46.381415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee2eeff3-db81-4da1-9361-55fae10d4310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:46.381911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee2eeff3-db81-4da1-9361-55fae10d4310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:46.382812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee2eeff3-db81-4da1-9361-55fae10d4310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:46.383283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee2eeff3-db81-4da1-9361-55fae10d4310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:46.519538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ee2eeff3-db81-4da1-9361-55fae10d4310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:46.529606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ee2eeff3-db81-4da1-9361-55fae10d4310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.159462 May 25 16:43:46.530224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1179/2336] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:46 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:47.151600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:47.152490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:47.161511 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:47.162111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:47.163122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:47.164163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:47.226456 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:47.235211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:47.236738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:47.237882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-73d8eba4-773b-4e23-9121-1519d6ddc9b3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.091563 May 25 16:43:47.238423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1158/2337] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:47 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:47.266385 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2d2c4ef5-03fa-4bcd-b683-db5176175f9d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:47.266998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2d2c4ef5-03fa-4bcd-b683-db5176175f9d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:47.278070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d2c4ef5-03fa-4bcd-b683-db5176175f9d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:47.278522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d2c4ef5-03fa-4bcd-b683-db5176175f9d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:47.279315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d2c4ef5-03fa-4bcd-b683-db5176175f9d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:47.279755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2d2c4ef5-03fa-4bcd-b683-db5176175f9d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:47.351558 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2d2c4ef5-03fa-4bcd-b683-db5176175f9d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:47.360699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2d2c4ef5-03fa-4bcd-b683-db5176175f9d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.098238 May 25 16:43:47.361281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1180/2338] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:47 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:47.390303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-12b28b21-d36e-4d34-b2dc-9b01e931336a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:47.390939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-12b28b21-d36e-4d34-b2dc-9b01e931336a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:47.397252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-12b28b21-d36e-4d34-b2dc-9b01e931336a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:47.397704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-12b28b21-d36e-4d34-b2dc-9b01e931336a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:47.398561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-12b28b21-d36e-4d34-b2dc-9b01e931336a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:47.398990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-12b28b21-d36e-4d34-b2dc-9b01e931336a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:47.536557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-12b28b21-d36e-4d34-b2dc-9b01e931336a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:47.545337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-12b28b21-d36e-4d34-b2dc-9b01e931336a tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.158956 May 25 16:43:47.545907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1159/2339] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:47 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:47.553475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-245ca892-945a-492c-8048-5af13e60dc56 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:47.554090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-245ca892-945a-492c-8048-5af13e60dc56 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:47.560517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-245ca892-945a-492c-8048-5af13e60dc56 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:47.560964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-245ca892-945a-492c-8048-5af13e60dc56 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:47.561702 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-245ca892-945a-492c-8048-5af13e60dc56 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:47.562153 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-245ca892-945a-492c-8048-5af13e60dc56 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:47.709410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-245ca892-945a-492c-8048-5af13e60dc56 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:47.717894 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-245ca892-945a-492c-8048-5af13e60dc56 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.168517 May 25 16:43:47.718467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1181/2340] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:47 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:48.261068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:48.262073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:48.273301 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:48.273758 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:48.274520 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:48.274952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:48.347313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:48.356054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:48.357559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:48.358698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-52479354-4e5f-4248-bf78-9cc659fc6ac6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.104274 May 25 16:43:48.359277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1160/2341] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:48 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:48.377696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f5e75375-7b58-487a-859a-b94383e23e93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:48.378307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f5e75375-7b58-487a-859a-b94383e23e93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:48.384846 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5e75375-7b58-487a-859a-b94383e23e93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:48.385296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5e75375-7b58-487a-859a-b94383e23e93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:48.386047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5e75375-7b58-487a-859a-b94383e23e93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:48.386478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f5e75375-7b58-487a-859a-b94383e23e93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:48.450988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f5e75375-7b58-487a-859a-b94383e23e93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:48.459493 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f5e75375-7b58-487a-859a-b94383e23e93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085643 May 25 16:43:48.460078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1182/2342] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:48 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:48.565974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0a27ea36-cfa3-4a3a-8b76-4fb140a80778 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:48.567055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0a27ea36-cfa3-4a3a-8b76-4fb140a80778 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:48.574528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a27ea36-cfa3-4a3a-8b76-4fb140a80778 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:48.574978 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a27ea36-cfa3-4a3a-8b76-4fb140a80778 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:48.575771 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a27ea36-cfa3-4a3a-8b76-4fb140a80778 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:48.576197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0a27ea36-cfa3-4a3a-8b76-4fb140a80778 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:48.727209 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0a27ea36-cfa3-4a3a-8b76-4fb140a80778 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:48.734275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-97f3db50-116a-4909-a7e8-a75fb043871b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:48.734887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-97f3db50-116a-4909-a7e8-a75fb043871b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:48.735858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0a27ea36-cfa3-4a3a-8b76-4fb140a80778 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.176444 May 25 16:43:48.736419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1161/2343] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:48 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:48.741528 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97f3db50-116a-4909-a7e8-a75fb043871b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:48.742010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97f3db50-116a-4909-a7e8-a75fb043871b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:48.742779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97f3db50-116a-4909-a7e8-a75fb043871b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:48.743217 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97f3db50-116a-4909-a7e8-a75fb043871b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:48.876364 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-97f3db50-116a-4909-a7e8-a75fb043871b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:48.888776 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-97f3db50-116a-4909-a7e8-a75fb043871b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.157964 May 25 16:43:48.889592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1183/2344] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:48 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:43:49.380888 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:49.381762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:49.389328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:49.389800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:49.390597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:49.391085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:49.406857 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:49.453925 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:49.462699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:49.464338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:49.465468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-968c4fb5-b42e-4651-9e2f-59882c9ef60d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.091202 May 25 16:43:49.466035 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1162/2345] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:49 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:49.484616 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9ff9a8a2-a465-403b-9947-e34d11871df6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:49.485234 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9ff9a8a2-a465-403b-9947-e34d11871df6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:49.495548 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ff9a8a2-a465-403b-9947-e34d11871df6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:49.496252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ff9a8a2-a465-403b-9947-e34d11871df6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:49.497273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ff9a8a2-a465-403b-9947-e34d11871df6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:49.497935 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9ff9a8a2-a465-403b-9947-e34d11871df6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:49.563669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9ff9a8a2-a465-403b-9947-e34d11871df6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:49.571970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9ff9a8a2-a465-403b-9947-e34d11871df6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093642 May 25 16:43:49.572551 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1184/2346] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:49 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:49.757799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6e8ab24f-78ef-42d0-a26c-5a60314e3291 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:49.758719 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6e8ab24f-78ef-42d0-a26c-5a60314e3291 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:49.769223 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e8ab24f-78ef-42d0-a26c-5a60314e3291 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:49.769980 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e8ab24f-78ef-42d0-a26c-5a60314e3291 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:49.771238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e8ab24f-78ef-42d0-a26c-5a60314e3291 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:49.771993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e8ab24f-78ef-42d0-a26c-5a60314e3291 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:49.901111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6e8ab24f-78ef-42d0-a26c-5a60314e3291 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:49.909038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-23e07976-d2d6-41a3-b283-0ca737027b97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:49.909256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6e8ab24f-78ef-42d0-a26c-5a60314e3291 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.158233 May 25 16:43:49.909656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-23e07976-d2d6-41a3-b283-0ca737027b97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:49.910182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1163/2347] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:49 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:49.916951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23e07976-d2d6-41a3-b283-0ca737027b97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:49.917402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23e07976-d2d6-41a3-b283-0ca737027b97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:49.918173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23e07976-d2d6-41a3-b283-0ca737027b97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:49.918603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-23e07976-d2d6-41a3-b283-0ca737027b97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:50.063250 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-23e07976-d2d6-41a3-b283-0ca737027b97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:50.071912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-23e07976-d2d6-41a3-b283-0ca737027b97 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.166794 May 25 16:43:50.072500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1185/2348] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:49 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:50.488028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:50.488028 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:50.496485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:50.496934 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:50.497688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:50.498123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:50.573914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:50.582937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:50.584485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:50.585712 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aae40cb3-09f1-4a2d-bb1d-939a33a907aa tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.105200 May 25 16:43:50.586245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1164/2349] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:50 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:50.588979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9492aa39-898f-49e1-b851-6ab1fa0c0b91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:50.589650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9492aa39-898f-49e1-b851-6ab1fa0c0b91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:50.595842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9492aa39-898f-49e1-b851-6ab1fa0c0b91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:50.596300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9492aa39-898f-49e1-b851-6ab1fa0c0b91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:50.597062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9492aa39-898f-49e1-b851-6ab1fa0c0b91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:50.597497 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9492aa39-898f-49e1-b851-6ab1fa0c0b91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:50.664736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9492aa39-898f-49e1-b851-6ab1fa0c0b91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:50.677782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9492aa39-898f-49e1-b851-6ab1fa0c0b91 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092437 May 25 16:43:50.678563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1186/2350] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:50 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:50.931054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3522e7ef-09bb-437a-a773-c2df173dfe7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:50.931986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3522e7ef-09bb-437a-a773-c2df173dfe7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:50.941970 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3522e7ef-09bb-437a-a773-c2df173dfe7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:50.942733 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3522e7ef-09bb-437a-a773-c2df173dfe7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:50.944046 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3522e7ef-09bb-437a-a773-c2df173dfe7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:50.944773 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3522e7ef-09bb-437a-a773-c2df173dfe7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:51.074148 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3522e7ef-09bb-437a-a773-c2df173dfe7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:51.082375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3522e7ef-09bb-437a-a773-c2df173dfe7d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.157900 May 25 16:43:51.082937 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1165/2351] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:50 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:51.089233 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-97c64b29-119a-4c0a-8d0a-1de74d8e3cc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:51.089847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-97c64b29-119a-4c0a-8d0a-1de74d8e3cc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:51.096478 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97c64b29-119a-4c0a-8d0a-1de74d8e3cc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:51.096927 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97c64b29-119a-4c0a-8d0a-1de74d8e3cc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:51.097678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97c64b29-119a-4c0a-8d0a-1de74d8e3cc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:51.098114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-97c64b29-119a-4c0a-8d0a-1de74d8e3cc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:51.230313 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-97c64b29-119a-4c0a-8d0a-1de74d8e3cc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:51.238426 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-97c64b29-119a-4c0a-8d0a-1de74d8e3cc2 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.153276 May 25 16:43:51.239000 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1187/2352] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:51 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:51.605681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:51.606519 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:51.616637 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:51.617347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:51.618604 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:51.619440 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:51.698486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:51.700147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7510942f-789b-4434-a1ab-d7b24f6afce7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:51.700782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7510942f-789b-4434-a1ab-d7b24f6afce7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:51.707081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7510942f-789b-4434-a1ab-d7b24f6afce7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:51.707563 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7510942f-789b-4434-a1ab-d7b24f6afce7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:51.707891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:51.708342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7510942f-789b-4434-a1ab-d7b24f6afce7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:51.708770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7510942f-789b-4434-a1ab-d7b24f6afce7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:51.709438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:51.710731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-de66935c-a0fd-458e-9fc5-05d2a0387ed3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.110371 May 25 16:43:51.711302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1166/2353] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:51 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:51.725214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:51.779931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7510942f-789b-4434-a1ab-d7b24f6afce7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:51.788320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7510942f-789b-4434-a1ab-d7b24f6afce7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.092112 May 25 16:43:51.789332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1188/2354] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:51 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:43:52.100354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-53b2fc79-5a19-4d77-bc4b-6d7d1529ff86 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:52.100693 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-53b2fc79-5a19-4d77-bc4b-6d7d1529ff86 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:52.106994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53b2fc79-5a19-4d77-bc4b-6d7d1529ff86 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:52.107461 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53b2fc79-5a19-4d77-bc4b-6d7d1529ff86 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:52.108225 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53b2fc79-5a19-4d77-bc4b-6d7d1529ff86 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:52.108705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-53b2fc79-5a19-4d77-bc4b-6d7d1529ff86 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:52.239845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-53b2fc79-5a19-4d77-bc4b-6d7d1529ff86 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:52.248266 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-53b2fc79-5a19-4d77-bc4b-6d7d1529ff86 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.152065 May 25 16:43:52.248843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1167/2355] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:52 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:52.255989 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d8d90dbc-d2b7-4829-a3de-2d23cb94963c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:52.256704 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d8d90dbc-d2b7-4829-a3de-2d23cb94963c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:52.263423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8d90dbc-d2b7-4829-a3de-2d23cb94963c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:52.263874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8d90dbc-d2b7-4829-a3de-2d23cb94963c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:52.265120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8d90dbc-d2b7-4829-a3de-2d23cb94963c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:52.265120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d8d90dbc-d2b7-4829-a3de-2d23cb94963c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:52.400132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d8d90dbc-d2b7-4829-a3de-2d23cb94963c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:52.410316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d8d90dbc-d2b7-4829-a3de-2d23cb94963c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.158285 May 25 16:43:52.410917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1189/2356] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:52 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:52.735008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:52.736067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:52.744993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:52.745438 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:52.746187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:52.746617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:52.806168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1e3c038a-b2f9-44b1-8f53-831d6045c72d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:52.806815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1e3c038a-b2f9-44b1-8f53-831d6045c72d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:52.813447 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1e3c038a-b2f9-44b1-8f53-831d6045c72d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:52.813900 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1e3c038a-b2f9-44b1-8f53-831d6045c72d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:52.814669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1e3c038a-b2f9-44b1-8f53-831d6045c72d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:52.815104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1e3c038a-b2f9-44b1-8f53-831d6045c72d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:52.816751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:52.826004 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:52.827524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:52.828678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-41b8a86e-55c4-4c60-a6d4-fcd7904557f3 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.101962 May 25 16:43:52.829238 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1168/2357] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:52 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:52.878375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1e3c038a-b2f9-44b1-8f53-831d6045c72d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:52.888373 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1e3c038a-b2f9-44b1-8f53-831d6045c72d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.086261 May 25 16:43:52.888943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1190/2358] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:52 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:53.269814 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-eedb1b33-0abe-4259-a55d-2e32ef7c9f2c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:53.270876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-eedb1b33-0abe-4259-a55d-2e32ef7c9f2c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:53.278522 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eedb1b33-0abe-4259-a55d-2e32ef7c9f2c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:53.278976 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eedb1b33-0abe-4259-a55d-2e32ef7c9f2c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:53.279797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eedb1b33-0abe-4259-a55d-2e32ef7c9f2c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:53.280213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-eedb1b33-0abe-4259-a55d-2e32ef7c9f2c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:53.426837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-eedb1b33-0abe-4259-a55d-2e32ef7c9f2c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:53.430591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-72f23065-8a55-46df-a936-113f43d3178e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:53.431365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-72f23065-8a55-46df-a936-113f43d3178e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:53.436007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-eedb1b33-0abe-4259-a55d-2e32ef7c9f2c tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.172725 May 25 16:43:53.436564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1169/2359] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:53 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:53.438071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-72f23065-8a55-46df-a936-113f43d3178e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:53.438524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-72f23065-8a55-46df-a936-113f43d3178e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:53.439315 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-72f23065-8a55-46df-a936-113f43d3178e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:53.439753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-72f23065-8a55-46df-a936-113f43d3178e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:53.569015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-72f23065-8a55-46df-a936-113f43d3178e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:53.577399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-72f23065-8a55-46df-a936-113f43d3178e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.152612 May 25 16:43:53.577976 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1191/2360] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:53 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:53.848332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:53.849098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:53.856471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:53.856942 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:53.857687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:53.858118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:53.909942 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-edafdff6-daa4-40db-82e5-d88e8e006715 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:53.910574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-edafdff6-daa4-40db-82e5-d88e8e006715 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:53.917396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edafdff6-daa4-40db-82e5-d88e8e006715 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:53.917845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edafdff6-daa4-40db-82e5-d88e8e006715 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:53.919643 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edafdff6-daa4-40db-82e5-d88e8e006715 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:53.920121 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-edafdff6-daa4-40db-82e5-d88e8e006715 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:53.920907 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:53.929869 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:53.931583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:53.932800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-05ae2a9e-7b0d-4a2c-8d33-9be4750545f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.089424 May 25 16:43:53.933459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1170/2361] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:53 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:53.988597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-edafdff6-daa4-40db-82e5-d88e8e006715 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:53.996830 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-edafdff6-daa4-40db-82e5-d88e8e006715 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090768 May 25 16:43:53.997450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1192/2362] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:53 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:54.456872 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ee8b890b-69d3-435b-8adb-b37ac0527008 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:54.457794 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ee8b890b-69d3-435b-8adb-b37ac0527008 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:54.467428 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee8b890b-69d3-435b-8adb-b37ac0527008 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:54.468116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee8b890b-69d3-435b-8adb-b37ac0527008 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:54.469320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee8b890b-69d3-435b-8adb-b37ac0527008 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:54.469987 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ee8b890b-69d3-435b-8adb-b37ac0527008 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:54.595405 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-48581d2a-4182-4c00-9d4a-13e2ba8d4cf8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:54.596170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-48581d2a-4182-4c00-9d4a-13e2ba8d4cf8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:54.603075 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-48581d2a-4182-4c00-9d4a-13e2ba8d4cf8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:54.603615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-48581d2a-4182-4c00-9d4a-13e2ba8d4cf8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:54.604442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-48581d2a-4182-4c00-9d4a-13e2ba8d4cf8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:54.604970 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-48581d2a-4182-4c00-9d4a-13e2ba8d4cf8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:54.605244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ee8b890b-69d3-435b-8adb-b37ac0527008 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:54.613572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ee8b890b-69d3-435b-8adb-b37ac0527008 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.162757 May 25 16:43:54.614145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1171/2363] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:54 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:54.737482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-48581d2a-4182-4c00-9d4a-13e2ba8d4cf8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:54.745898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-48581d2a-4182-4c00-9d4a-13e2ba8d4cf8 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.154884 May 25 16:43:54.746482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1193/2364] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:54 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:43:54.955016 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:54.955586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:54.962656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:54.963108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:54.963905 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:54.964376 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:55.013969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-543fe9e3-1735-411f-a5c5-adfef0983d4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:55.014568 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-543fe9e3-1735-411f-a5c5-adfef0983d4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:55.020742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-543fe9e3-1735-411f-a5c5-adfef0983d4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:55.021189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-543fe9e3-1735-411f-a5c5-adfef0983d4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:55.021948 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-543fe9e3-1735-411f-a5c5-adfef0983d4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:55.022382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-543fe9e3-1735-411f-a5c5-adfef0983d4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:55.031887 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:55.044314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:55.045964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:55.047261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e5adfd6d-1dde-4af0-9048-79b02d131b45 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.096251 May 25 16:43:55.047913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1172/2365] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:54 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:55.084096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-543fe9e3-1735-411f-a5c5-adfef0983d4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:55.092272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-543fe9e3-1735-411f-a5c5-adfef0983d4d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.082216 May 25 16:43:55.092845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1194/2366] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:55 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:55.635021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-55594c77-87f1-45e4-b4ee-951f5e4c5542 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:55.636092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-55594c77-87f1-45e4-b4ee-951f5e4c5542 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:55.646219 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55594c77-87f1-45e4-b4ee-951f5e4c5542 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:55.646915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55594c77-87f1-45e4-b4ee-951f5e4c5542 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:55.648394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55594c77-87f1-45e4-b4ee-951f5e4c5542 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:55.648825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-55594c77-87f1-45e4-b4ee-951f5e4c5542 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:55.763572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-75518c8b-4dd9-4d9f-80b8-c86d5ebefb67 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:55.764209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-75518c8b-4dd9-4d9f-80b8-c86d5ebefb67 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:55.771053 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75518c8b-4dd9-4d9f-80b8-c86d5ebefb67 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:55.771540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75518c8b-4dd9-4d9f-80b8-c86d5ebefb67 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:55.772307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75518c8b-4dd9-4d9f-80b8-c86d5ebefb67 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:55.772734 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75518c8b-4dd9-4d9f-80b8-c86d5ebefb67 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:55.783922 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-55594c77-87f1-45e4-b4ee-951f5e4c5542 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:55.792244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-55594c77-87f1-45e4-b4ee-951f5e4c5542 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.163851 May 25 16:43:55.792813 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1173/2367] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:55 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:55.910132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-75518c8b-4dd9-4d9f-80b8-c86d5ebefb67 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:55.920923 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-75518c8b-4dd9-4d9f-80b8-c86d5ebefb67 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.161314 May 25 16:43:55.921506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1195/2368] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:55 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:56.073372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:56.073372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:56.081266 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:56.081728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:56.082501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:56.082935 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:56.110370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-901b8fa7-8c06-4dae-a2b0-ee3412ab2fc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:56.110936 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-901b8fa7-8c06-4dae-a2b0-ee3412ab2fc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:56.120593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-901b8fa7-8c06-4dae-a2b0-ee3412ab2fc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:56.121052 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-901b8fa7-8c06-4dae-a2b0-ee3412ab2fc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:56.121810 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-901b8fa7-8c06-4dae-a2b0-ee3412ab2fc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:56.122245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-901b8fa7-8c06-4dae-a2b0-ee3412ab2fc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:56.146317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:56.155307 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:56.156892 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:56.158099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7c0afd5d-ccc6-4e12-8fd9-b5ee793c8eb1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.094723 May 25 16:43:56.158723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1174/2369] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:56 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:56.183263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-901b8fa7-8c06-4dae-a2b0-ee3412ab2fc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:56.191757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-901b8fa7-8c06-4dae-a2b0-ee3412ab2fc1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085338 May 25 16:43:56.192331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1196/2370] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:56 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:56.811271 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-28bed509-b9cc-46b9-a16f-0d9d325ad270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:56.812022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-28bed509-b9cc-46b9-a16f-0d9d325ad270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:56.819871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28bed509-b9cc-46b9-a16f-0d9d325ad270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:56.820294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28bed509-b9cc-46b9-a16f-0d9d325ad270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:56.821050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28bed509-b9cc-46b9-a16f-0d9d325ad270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:56.821485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28bed509-b9cc-46b9-a16f-0d9d325ad270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:56.942781 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d0048187-0422-4228-96cb-31670ba7dc26 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:56.943453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d0048187-0422-4228-96cb-31670ba7dc26 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:56.950403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0048187-0422-4228-96cb-31670ba7dc26 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:56.950860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0048187-0422-4228-96cb-31670ba7dc26 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:56.951653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0048187-0422-4228-96cb-31670ba7dc26 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:56.952090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d0048187-0422-4228-96cb-31670ba7dc26 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:56.962445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-28bed509-b9cc-46b9-a16f-0d9d325ad270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:56.970666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-28bed509-b9cc-46b9-a16f-0d9d325ad270 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.164214 May 25 16:43:56.971242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1175/2371] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:56 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:57.085468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d0048187-0422-4228-96cb-31670ba7dc26 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:57.093676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d0048187-0422-4228-96cb-31670ba7dc26 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.154780 May 25 16:43:57.094275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1197/2372] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:56 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:57.180383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:57.181085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:57.189186 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:57.189720 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:57.190615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:57.191142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:57.209916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a1efecff-df37-4ed2-81fb-b4548ae747e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:57.210744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a1efecff-df37-4ed2-81fb-b4548ae747e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:57.216902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1efecff-df37-4ed2-81fb-b4548ae747e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:57.217354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1efecff-df37-4ed2-81fb-b4548ae747e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:57.218115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1efecff-df37-4ed2-81fb-b4548ae747e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:57.218553 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a1efecff-df37-4ed2-81fb-b4548ae747e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:57.257798 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:57.266698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:57.268320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:57.269464 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-190501d1-8655-44b1-baab-3c198b503e10 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.095342 May 25 16:43:57.270057 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1176/2373] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:57 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:57.286949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a1efecff-df37-4ed2-81fb-b4548ae747e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:57.295664 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a1efecff-df37-4ed2-81fb-b4548ae747e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.090260 May 25 16:43:57.296241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1198/2374] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:57 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:57.992580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d5b9036e-6e9a-4316-b86f-206f48abe0cc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:57.993296 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d5b9036e-6e9a-4316-b86f-206f48abe0cc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:58.000777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5b9036e-6e9a-4316-b86f-206f48abe0cc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:58.001305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5b9036e-6e9a-4316-b86f-206f48abe0cc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:58.002180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5b9036e-6e9a-4316-b86f-206f48abe0cc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:58.002681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d5b9036e-6e9a-4316-b86f-206f48abe0cc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:58.113008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-344a6d98-4b34-49c6-b18f-19268213d859 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:58.113631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-344a6d98-4b34-49c6-b18f-19268213d859 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:58.120093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-344a6d98-4b34-49c6-b18f-19268213d859 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:58.120545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-344a6d98-4b34-49c6-b18f-19268213d859 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:58.121310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-344a6d98-4b34-49c6-b18f-19268213d859 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:58.121784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-344a6d98-4b34-49c6-b18f-19268213d859 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:58.136914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d5b9036e-6e9a-4316-b86f-206f48abe0cc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:58.145151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d5b9036e-6e9a-4316-b86f-206f48abe0cc tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.159133 May 25 16:43:58.145724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1177/2375] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:57 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:58.259555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-344a6d98-4b34-49c6-b18f-19268213d859 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:58.272506 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-344a6d98-4b34-49c6-b18f-19268213d859 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.164317 May 25 16:43:58.273556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1199/2376] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:58 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:58.291022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:58.291663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:58.298079 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:58.298531 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:58.299328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:58.299762 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:58.312682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-13b64acf-51e7-4da3-a1b9-f745594fdcbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:58.313375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-13b64acf-51e7-4da3-a1b9-f745594fdcbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:58.319471 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13b64acf-51e7-4da3-a1b9-f745594fdcbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:58.320218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13b64acf-51e7-4da3-a1b9-f745594fdcbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:58.320971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13b64acf-51e7-4da3-a1b9-f745594fdcbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:58.321400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-13b64acf-51e7-4da3-a1b9-f745594fdcbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:58.366751 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:58.375932 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:58.377419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:58.378586 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4bca7c4c-600c-4274-8403-73dbfff00e06 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.093216 May 25 16:43:58.379117 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1178/2377] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:58 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:58.391993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-13b64acf-51e7-4da3-a1b9-f745594fdcbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:58.400225 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-13b64acf-51e7-4da3-a1b9-f745594fdcbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091381 May 25 16:43:58.400782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1200/2378] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:58 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:59.165774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-88e5b1d6-525a-4178-bfee-1bec582acbec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:59.166466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-88e5b1d6-525a-4178-bfee-1bec582acbec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:59.174617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88e5b1d6-525a-4178-bfee-1bec582acbec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:59.175145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88e5b1d6-525a-4178-bfee-1bec582acbec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:59.176080 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88e5b1d6-525a-4178-bfee-1bec582acbec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:59.176597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-88e5b1d6-525a-4178-bfee-1bec582acbec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:59.179611 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:43:59.295640 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c1e4d6d3-591d-4740-b82f-e82c86697d7a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:59.296267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c1e4d6d3-591d-4740-b82f-e82c86697d7a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:59.303564 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1e4d6d3-591d-4740-b82f-e82c86697d7a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:59.304227 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1e4d6d3-591d-4740-b82f-e82c86697d7a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:59.304883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1e4d6d3-591d-4740-b82f-e82c86697d7a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:59.305311 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c1e4d6d3-591d-4740-b82f-e82c86697d7a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:59.317281 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-88e5b1d6-525a-4178-bfee-1bec582acbec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:59.326543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-88e5b1d6-525a-4178-bfee-1bec582acbec tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.166992 May 25 16:43:59.327023 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1179/2379] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:59 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:59.398358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:59.399295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:59.406263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:59.406736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:59.407622 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:59.408101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:59.465862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c1e4d6d3-591d-4740-b82f-e82c86697d7a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:59.474468 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c1e4d6d3-591d-4740-b82f-e82c86697d7a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.182850 May 25 16:43:59.475047 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1201/2380] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:59 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:43:59.481129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:59.481482 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-bdc40c12-ae8a-4d4a-b698-70a1ee26aeba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:43:59.482099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-bdc40c12-ae8a-4d4a-b698-70a1ee26aeba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:43:59.488334 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bdc40c12-ae8a-4d4a-b698-70a1ee26aeba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:59.488784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bdc40c12-ae8a-4d4a-b698-70a1ee26aeba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:59.489549 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bdc40c12-ae8a-4d4a-b698-70a1ee26aeba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:43:59.490019 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-bdc40c12-ae8a-4d4a-b698-70a1ee26aeba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:43:59.490210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:59.491724 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:59.492862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a6d68c10-fe97-440e-9271-e947e31c4687 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.099987 May 25 16:43:59.493426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1180/2381] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:43:59 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:43:59.551990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-bdc40c12-ae8a-4d4a-b698-70a1ee26aeba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:43:59.560205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-bdc40c12-ae8a-4d4a-b698-70a1ee26aeba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084151 May 25 16:43:59.560771 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1202/2382] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:43:59 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:00.348284 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c9168dc2-c327-4317-8d8a-4aef516e301b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:00.349165 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c9168dc2-c327-4317-8d8a-4aef516e301b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:00.360086 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9168dc2-c327-4317-8d8a-4aef516e301b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:00.360843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9168dc2-c327-4317-8d8a-4aef516e301b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:00.362094 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9168dc2-c327-4317-8d8a-4aef516e301b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:00.362837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c9168dc2-c327-4317-8d8a-4aef516e301b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:00.492592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-61ae7a57-456e-44e1-ae6c-81e0585f38bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:00.493188 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-61ae7a57-456e-44e1-ae6c-81e0585f38bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:00.497898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c9168dc2-c327-4317-8d8a-4aef516e301b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:00.500157 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-61ae7a57-456e-44e1-ae6c-81e0585f38bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:00.500613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-61ae7a57-456e-44e1-ae6c-81e0585f38bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:00.501384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-61ae7a57-456e-44e1-ae6c-81e0585f38bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:00.501824 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-61ae7a57-456e-44e1-ae6c-81e0585f38bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:00.506635 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c9168dc2-c327-4317-8d8a-4aef516e301b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.164921 May 25 16:44:00.507478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1181/2383] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:00 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:00.512182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:00.512792 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:00.518702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:00.519144 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:00.519918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:00.520342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:00.596931 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:00.605994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:00.607458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:00.608601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-97db891f-8a20-4979-ad41-d03559faaf4e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.099996 May 25 16:44:00.609775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1182/2384] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:00 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:00.613496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-57d83bc8-469d-4992-8688-916a9ca74dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:00.614215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-57d83bc8-469d-4992-8688-916a9ca74dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:00.620601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-57d83bc8-469d-4992-8688-916a9ca74dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:00.621055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-57d83bc8-469d-4992-8688-916a9ca74dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:00.621815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-57d83bc8-469d-4992-8688-916a9ca74dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:00.622254 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-57d83bc8-469d-4992-8688-916a9ca74dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:00.671882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-61ae7a57-456e-44e1-ae6c-81e0585f38bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:00.680331 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-61ae7a57-456e-44e1-ae6c-81e0585f38bb tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.191665 May 25 16:44:00.680882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1203/2385] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:00 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:00.687682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-57d83bc8-469d-4992-8688-916a9ca74dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:00.695956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-57d83bc8-469d-4992-8688-916a9ca74dcc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085852 May 25 16:44:00.696517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1183/2386] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:00 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:01.528467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9448cb6f-7ff1-41df-b835-89e78e936390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:01.529450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9448cb6f-7ff1-41df-b835-89e78e936390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:01.540797 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9448cb6f-7ff1-41df-b835-89e78e936390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:01.541555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9448cb6f-7ff1-41df-b835-89e78e936390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:01.542817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9448cb6f-7ff1-41df-b835-89e78e936390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:01.543699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9448cb6f-7ff1-41df-b835-89e78e936390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:01.626149 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:01.626739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:01.633745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:01.634191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:01.634938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:01.635416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:01.687115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9448cb6f-7ff1-41df-b835-89e78e936390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:01.695669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9448cb6f-7ff1-41df-b835-89e78e936390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.173676 May 25 16:44:01.696213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1204/2387] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:01 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:01.700626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fba0af92-f52d-4fca-972a-182e81e2a20f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:01.701232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fba0af92-f52d-4fca-972a-182e81e2a20f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:01.703171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:01.707309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fba0af92-f52d-4fca-972a-182e81e2a20f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:01.707871 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fba0af92-f52d-4fca-972a-182e81e2a20f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:01.708639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fba0af92-f52d-4fca-972a-182e81e2a20f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:01.709077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fba0af92-f52d-4fca-972a-182e81e2a20f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:01.712548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:01.713160 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:01.714062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:01.715385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e0564dea-cf73-4069-a789-311f54a048a9 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.093081 May 25 16:44:01.715972 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1184/2388] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:01 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:01.720392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b96ada8d-1eb6-4ad1-b8a1-217ccac0a376 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:01.720994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b96ada8d-1eb6-4ad1-b8a1-217ccac0a376 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:01.727091 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b96ada8d-1eb6-4ad1-b8a1-217ccac0a376 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:01.727675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b96ada8d-1eb6-4ad1-b8a1-217ccac0a376 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:01.728477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b96ada8d-1eb6-4ad1-b8a1-217ccac0a376 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:01.728915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b96ada8d-1eb6-4ad1-b8a1-217ccac0a376 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:01.803815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b96ada8d-1eb6-4ad1-b8a1-217ccac0a376 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:01.811996 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b96ada8d-1eb6-4ad1-b8a1-217ccac0a376 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.095037 May 25 16:44:01.812810 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1185/2389] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:01 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:01.866147 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fba0af92-f52d-4fca-972a-182e81e2a20f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:01.874436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fba0af92-f52d-4fca-972a-182e81e2a20f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.177256 May 25 16:44:01.875000 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1205/2390] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:01 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:02.717098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3f0fb963-f368-4a1f-9b11-3cf0f602e3f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:02.718760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3f0fb963-f368-4a1f-9b11-3cf0f602e3f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:02.727992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3f0fb963-f368-4a1f-9b11-3cf0f602e3f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:02.728430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3f0fb963-f368-4a1f-9b11-3cf0f602e3f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:02.729210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3f0fb963-f368-4a1f-9b11-3cf0f602e3f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:02.729679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3f0fb963-f368-4a1f-9b11-3cf0f602e3f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:02.733327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:02.733963 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:02.740257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:02.740706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:02.741470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:02.741904 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:02.830132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:02.839132 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:02.840637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:02.841770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e76d54f7-3183-4cd3-a3de-57a8c5a52a1d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.112579 May 25 16:44:02.842309 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1206/2391] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:02 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:02.846626 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0e92c3c6-d0f8-46c6-9fb1-29eb7eda4b01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:02.847248 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0e92c3c6-d0f8-46c6-9fb1-29eb7eda4b01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:02.853303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e92c3c6-d0f8-46c6-9fb1-29eb7eda4b01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:02.853775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e92c3c6-d0f8-46c6-9fb1-29eb7eda4b01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:02.854522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e92c3c6-d0f8-46c6-9fb1-29eb7eda4b01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:02.854944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0e92c3c6-d0f8-46c6-9fb1-29eb7eda4b01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:02.907956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3f0fb963-f368-4a1f-9b11-3cf0f602e3f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:02.916788 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3f0fb963-f368-4a1f-9b11-3cf0f602e3f0 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.206537 May 25 16:44:02.917570 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1186/2392] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:02 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:02.921816 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3fd860be-75df-4315-8e8d-c3dbfce91e8f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:02.922415 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3fd860be-75df-4315-8e8d-c3dbfce91e8f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:02.926175 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0e92c3c6-d0f8-46c6-9fb1-29eb7eda4b01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:02.928485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3fd860be-75df-4315-8e8d-c3dbfce91e8f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:02.928938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3fd860be-75df-4315-8e8d-c3dbfce91e8f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:02.929692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3fd860be-75df-4315-8e8d-c3dbfce91e8f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:02.930125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3fd860be-75df-4315-8e8d-c3dbfce91e8f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:02.934491 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0e92c3c6-d0f8-46c6-9fb1-29eb7eda4b01 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.091258 May 25 16:44:02.935089 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1207/2393] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:02 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:03.069870 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3fd860be-75df-4315-8e8d-c3dbfce91e8f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:03.077561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3fd860be-75df-4315-8e8d-c3dbfce91e8f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.159221 May 25 16:44:03.078128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1187/2394] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:02 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:03.351230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:44:03.351590 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:44:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:44:03.863719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:03.864660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:03.872393 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:03.872842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:03.873598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:03.874027 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:03.934287 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a7e116b4-27e0-4c18-b2bb-3d5902bf1914 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:03.934928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a7e116b4-27e0-4c18-b2bb-3d5902bf1914 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:03.936856 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:03.941218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a7e116b4-27e0-4c18-b2bb-3d5902bf1914 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:03.941655 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a7e116b4-27e0-4c18-b2bb-3d5902bf1914 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:03.942410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a7e116b4-27e0-4c18-b2bb-3d5902bf1914 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:03.943067 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a7e116b4-27e0-4c18-b2bb-3d5902bf1914 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:03.945479 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:03.947022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:03.948200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-aff4a747-adfb-427e-ad92-03c039579f7d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.091075 May 25 16:44:03.948749 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1208/2395] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:03 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:03.953114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-357193a6-4082-4b11-be9a-eb8c0a363b8c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:03.953721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-357193a6-4082-4b11-be9a-eb8c0a363b8c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:03.959712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-357193a6-4082-4b11-be9a-eb8c0a363b8c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:03.960209 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-357193a6-4082-4b11-be9a-eb8c0a363b8c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:03.960998 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-357193a6-4082-4b11-be9a-eb8c0a363b8c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:03.961417 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-357193a6-4082-4b11-be9a-eb8c0a363b8c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:04.041780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-357193a6-4082-4b11-be9a-eb8c0a363b8c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:04.050028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-357193a6-4082-4b11-be9a-eb8c0a363b8c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.100322 May 25 16:44:04.050629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1209/2396] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:03 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:04.087863 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:44:04.088068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:44:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a (ip 10.222.0.45) !!! May 25 16:44:04.095819 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e41555d9-35a1-4248-aac7-5f9b1c7c271b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:04.096503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e41555d9-35a1-4248-aac7-5f9b1c7c271b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:04.099311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a7e116b4-27e0-4c18-b2bb-3d5902bf1914 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:04.102793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e41555d9-35a1-4248-aac7-5f9b1c7c271b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:04.103433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e41555d9-35a1-4248-aac7-5f9b1c7c271b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:04.104205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e41555d9-35a1-4248-aac7-5f9b1c7c271b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:04.104650 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e41555d9-35a1-4248-aac7-5f9b1c7c271b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:04.107638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a7e116b4-27e0-4c18-b2bb-3d5902bf1914 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.177347 May 25 16:44:04.108266 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1188/2397] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:03 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:04.238092 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e41555d9-35a1-4248-aac7-5f9b1c7c271b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:04.246915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e41555d9-35a1-4248-aac7-5f9b1c7c271b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.155803 May 25 16:44:04.247540 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1210/2398] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:04 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:04.970055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:04.971036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:04.980167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:04.980607 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:04.981360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:04.981800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:05.052487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:05.061476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:05.063367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:05.064803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6093f79f-d2eb-4ae0-8be7-a3f67144106c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.101315 May 25 16:44:05.065560 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1189/2399] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:04 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:05.069216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7a1da2f4-4520-41a8-881b-554af3c9da57 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:05.069775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-7a1da2f4-4520-41a8-881b-554af3c9da57 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:05.076854 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1da2f4-4520-41a8-881b-554af3c9da57 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:05.077310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1da2f4-4520-41a8-881b-554af3c9da57 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:05.078081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1da2f4-4520-41a8-881b-554af3c9da57 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:05.078523 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-7a1da2f4-4520-41a8-881b-554af3c9da57 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:05.125252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1bc44eca-7df7-4688-8c22-97917125ca83 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:05.125953 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1bc44eca-7df7-4688-8c22-97917125ca83 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:05.132041 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1bc44eca-7df7-4688-8c22-97917125ca83 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:05.132527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1bc44eca-7df7-4688-8c22-97917125ca83 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:05.133336 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1bc44eca-7df7-4688-8c22-97917125ca83 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:05.133815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1bc44eca-7df7-4688-8c22-97917125ca83 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:05.144303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-7a1da2f4-4520-41a8-881b-554af3c9da57 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:05.152670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7a1da2f4-4520-41a8-881b-554af3c9da57 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.089397 May 25 16:44:05.153272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1211/2400] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:05 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:44:05.264496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-672763f2-158c-42e3-9fbf-3904bf86f60c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:05.265391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-672763f2-158c-42e3-9fbf-3904bf86f60c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:05.267493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1bc44eca-7df7-4688-8c22-97917125ca83 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:05.271796 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-672763f2-158c-42e3-9fbf-3904bf86f60c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:05.272330 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-672763f2-158c-42e3-9fbf-3904bf86f60c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:05.273442 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-672763f2-158c-42e3-9fbf-3904bf86f60c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:05.273874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-672763f2-158c-42e3-9fbf-3904bf86f60c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:05.275903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1bc44eca-7df7-4688-8c22-97917125ca83 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.154526 May 25 16:44:05.276803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1190/2401] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:05 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:44:05.403898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-672763f2-158c-42e3-9fbf-3904bf86f60c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:05.416862 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-672763f2-158c-42e3-9fbf-3904bf86f60c tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.156064 May 25 16:44:05.417668 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1212/2402] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:05 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:44:06.086123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:06.086717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:06.093803 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:06.094276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:06.095005 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:06.095466 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:06.163398 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:06.172104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-06bad9a8-8a32-417e-a209-17c55cfec83d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:06.172908 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-06bad9a8-8a32-417e-a209-17c55cfec83d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:06.175205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:06.176745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:06.177889 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e453a7e5-8d40-480f-a11c-41985cd6d8e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098472 May 25 16:44:06.178416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1191/2403] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:06 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:06.180721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-06bad9a8-8a32-417e-a209-17c55cfec83d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:06.181166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-06bad9a8-8a32-417e-a209-17c55cfec83d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:06.181925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-06bad9a8-8a32-417e-a209-17c55cfec83d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:06.182352 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-06bad9a8-8a32-417e-a209-17c55cfec83d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:06.253772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-06bad9a8-8a32-417e-a209-17c55cfec83d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:06.261972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-06bad9a8-8a32-417e-a209-17c55cfec83d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.095811 May 25 16:44:06.262531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1213/2404] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:06 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:06.296443 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-446f7600-6d33-47e8-897c-4e5e853a6709 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:06.297263 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-446f7600-6d33-47e8-897c-4e5e853a6709 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:06.305602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-446f7600-6d33-47e8-897c-4e5e853a6709 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:06.306104 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-446f7600-6d33-47e8-897c-4e5e853a6709 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:06.306826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-446f7600-6d33-47e8-897c-4e5e853a6709 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:06.307303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-446f7600-6d33-47e8-897c-4e5e853a6709 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:06.438347 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-446f7600-6d33-47e8-897c-4e5e853a6709 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:06.439509 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2dfa65be-6c0a-426b-9791-be789c1fb310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:06.440118 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2dfa65be-6c0a-426b-9791-be789c1fb310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:06.446287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2dfa65be-6c0a-426b-9791-be789c1fb310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:06.446479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-446f7600-6d33-47e8-897c-4e5e853a6709 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.156134 May 25 16:44:06.446732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2dfa65be-6c0a-426b-9791-be789c1fb310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:06.447068 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1192/2405] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:06 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:06.447520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2dfa65be-6c0a-426b-9791-be789c1fb310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:06.447941 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2dfa65be-6c0a-426b-9791-be789c1fb310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:06.577100 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2dfa65be-6c0a-426b-9791-be789c1fb310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:06.589848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2dfa65be-6c0a-426b-9791-be789c1fb310 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.154169 May 25 16:44:06.590661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1214/2406] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:06 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:07.200957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:07.202000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:07.212866 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:07.213314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:07.214426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:07.214859 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:07.279610 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fa0c713f-bd01-4227-bc9d-84774ead466b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:07.280210 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fa0c713f-bd01-4227-bc9d-84774ead466b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:07.285380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:07.287287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fa0c713f-bd01-4227-bc9d-84774ead466b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:07.287744 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fa0c713f-bd01-4227-bc9d-84774ead466b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:07.288512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fa0c713f-bd01-4227-bc9d-84774ead466b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:07.289452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fa0c713f-bd01-4227-bc9d-84774ead466b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:07.294708 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:07.296211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:07.297338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-264bca15-8fb2-4e7a-85bb-8a4109eaefe5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.103572 May 25 16:44:07.297912 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1193/2407] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:07 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:07.350681 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-fa0c713f-bd01-4227-bc9d-84774ead466b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:07.358958 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fa0c713f-bd01-4227-bc9d-84774ead466b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.083332 May 25 16:44:07.359539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1215/2408] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:07 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:07.467863 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-220a6a7c-2cf0-4db5-95b4-8f5a31b89a04 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:07.468874 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-220a6a7c-2cf0-4db5-95b4-8f5a31b89a04 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:07.478848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-220a6a7c-2cf0-4db5-95b4-8f5a31b89a04 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:07.479407 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-220a6a7c-2cf0-4db5-95b4-8f5a31b89a04 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:07.480170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-220a6a7c-2cf0-4db5-95b4-8f5a31b89a04 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:07.480616 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-220a6a7c-2cf0-4db5-95b4-8f5a31b89a04 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:07.611480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e58ea354-f800-45ab-82a9-3356e45707fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:07.612087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e58ea354-f800-45ab-82a9-3356e45707fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:07.618451 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e58ea354-f800-45ab-82a9-3356e45707fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:07.618905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e58ea354-f800-45ab-82a9-3356e45707fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:07.619644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-220a6a7c-2cf0-4db5-95b4-8f5a31b89a04 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:07.619836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e58ea354-f800-45ab-82a9-3356e45707fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:07.620127 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e58ea354-f800-45ab-82a9-3356e45707fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:07.627975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-220a6a7c-2cf0-4db5-95b4-8f5a31b89a04 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.166758 May 25 16:44:07.628525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1194/2409] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:07 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:07.757536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e58ea354-f800-45ab-82a9-3356e45707fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:07.766271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e58ea354-f800-45ab-82a9-3356e45707fc tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.158736 May 25 16:44:07.766836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1216/2410] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:07 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:08.319584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:08.320559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:08.323550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:08.329102 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:08.329554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:08.330316 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:08.330749 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:08.375485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-55a15c4c-4ba9-40ca-8ca7-2bad937bfc3d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:08.376097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-55a15c4c-4ba9-40ca-8ca7-2bad937bfc3d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:08.383844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-55a15c4c-4ba9-40ca-8ca7-2bad937bfc3d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:08.384389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-55a15c4c-4ba9-40ca-8ca7-2bad937bfc3d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:08.385170 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-55a15c4c-4ba9-40ca-8ca7-2bad937bfc3d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:08.385607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-55a15c4c-4ba9-40ca-8ca7-2bad937bfc3d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:08.398732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:08.401062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:08.408775 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:08.410298 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:08.411495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-83b8fbeb-ea5d-4433-bf59-7c978bcc0f0c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098501 May 25 16:44:08.412269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1195/2411] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:08 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:08.451440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-55a15c4c-4ba9-40ca-8ca7-2bad937bfc3d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:08.459661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-55a15c4c-4ba9-40ca-8ca7-2bad937bfc3d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.088094 May 25 16:44:08.460195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1217/2412] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:08 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:08.649731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-44dcd444-9532-451e-a1a4-1c68ed989995 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:08.650568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-44dcd444-9532-451e-a1a4-1c68ed989995 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:08.657151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-44dcd444-9532-451e-a1a4-1c68ed989995 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:08.657626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-44dcd444-9532-451e-a1a4-1c68ed989995 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:08.658390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-44dcd444-9532-451e-a1a4-1c68ed989995 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:08.658826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-44dcd444-9532-451e-a1a4-1c68ed989995 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:08.783769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6ed938f5-a75b-477b-9980-93d20ff6f392 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:08.784378 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6ed938f5-a75b-477b-9980-93d20ff6f392 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:08.791303 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ed938f5-a75b-477b-9980-93d20ff6f392 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:08.791752 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ed938f5-a75b-477b-9980-93d20ff6f392 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:08.792501 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ed938f5-a75b-477b-9980-93d20ff6f392 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:08.792929 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6ed938f5-a75b-477b-9980-93d20ff6f392 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:08.797491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-44dcd444-9532-451e-a1a4-1c68ed989995 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:08.805880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-44dcd444-9532-451e-a1a4-1c68ed989995 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.162869 May 25 16:44:08.806509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1196/2413] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:08 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:08.945246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6ed938f5-a75b-477b-9980-93d20ff6f392 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:08.953572 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6ed938f5-a75b-477b-9980-93d20ff6f392 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.173657 May 25 16:44:08.954115 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1218/2414] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:08 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:09.433514 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:09.434143 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:09.441154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:09.441601 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:09.442360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:09.442787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:09.476706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e98a5dc0-2369-45f1-a712-34e1ccf864c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:09.477396 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e98a5dc0-2369-45f1-a712-34e1ccf864c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:09.484508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e98a5dc0-2369-45f1-a712-34e1ccf864c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:09.485044 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e98a5dc0-2369-45f1-a712-34e1ccf864c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:09.485916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e98a5dc0-2369-45f1-a712-34e1ccf864c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:09.486431 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e98a5dc0-2369-45f1-a712-34e1ccf864c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:09.505382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:09.514107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:09.515642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:09.516793 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3e4e1d75-4155-45f5-9792-5dbdd2559b53 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.088565 May 25 16:44:09.517383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1197/2415] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:09 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:09.550443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-e98a5dc0-2369-45f1-a712-34e1ccf864c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:09.558699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e98a5dc0-2369-45f1-a712-34e1ccf864c7 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.085894 May 25 16:44:09.559305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1219/2416] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:09 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:09.825517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b7f37aa0-56c5-4175-a8e4-bca78418c572 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:09.826118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b7f37aa0-56c5-4175-a8e4-bca78418c572 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:09.832530 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7f37aa0-56c5-4175-a8e4-bca78418c572 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:09.832973 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7f37aa0-56c5-4175-a8e4-bca78418c572 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:09.833716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7f37aa0-56c5-4175-a8e4-bca78418c572 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:09.834142 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7f37aa0-56c5-4175-a8e4-bca78418c572 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:09.971618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-64725e27-b5e7-4b0d-b261-05f377b3d100 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:09.972238 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-64725e27-b5e7-4b0d-b261-05f377b3d100 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:09.978214 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b7f37aa0-56c5-4175-a8e4-bca78418c572 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:09.978415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64725e27-b5e7-4b0d-b261-05f377b3d100 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:09.978849 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64725e27-b5e7-4b0d-b261-05f377b3d100 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:09.979642 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64725e27-b5e7-4b0d-b261-05f377b3d100 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:09.980081 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-64725e27-b5e7-4b0d-b261-05f377b3d100 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:09.986615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b7f37aa0-56c5-4175-a8e4-bca78418c572 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.165652 May 25 16:44:09.987249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1198/2417] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:09 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:10.111723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-64725e27-b5e7-4b0d-b261-05f377b3d100 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:10.119874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-64725e27-b5e7-4b0d-b261-05f377b3d100 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.152684 May 25 16:44:10.120420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1220/2418] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:09 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:10.539204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:10.540220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:10.552433 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:10.553177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:10.554491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:10.555197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:10.575624 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d015537b-7849-4b4e-a40b-e1f6c82e57c3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:10.576283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d015537b-7849-4b4e-a40b-e1f6c82e57c3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:10.583000 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d015537b-7849-4b4e-a40b-e1f6c82e57c3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:10.583470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d015537b-7849-4b4e-a40b-e1f6c82e57c3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:10.584215 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d015537b-7849-4b4e-a40b-e1f6c82e57c3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:10.584639 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d015537b-7849-4b4e-a40b-e1f6c82e57c3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:10.621300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:10.630273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:10.631797 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:10.633196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3424c90b-9b03-466e-907e-152403b7347d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.100905 May 25 16:44:10.633716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1199/2419] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:10 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:10.648926 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d015537b-7849-4b4e-a40b-e1f6c82e57c3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:10.651271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:10.658057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d015537b-7849-4b4e-a40b-e1f6c82e57c3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.086374 May 25 16:44:10.658605 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1221/2420] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:10 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:11.007218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b30493b6-721a-41e5-9dbe-42da0e60a76d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:11.008231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b30493b6-721a-41e5-9dbe-42da0e60a76d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:11.015508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b30493b6-721a-41e5-9dbe-42da0e60a76d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:11.015981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b30493b6-721a-41e5-9dbe-42da0e60a76d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:11.016770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b30493b6-721a-41e5-9dbe-42da0e60a76d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:11.017226 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b30493b6-721a-41e5-9dbe-42da0e60a76d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:11.138268 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3bc4a5c5-295b-4935-ae02-ed0c43f6faaf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:11.138875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3bc4a5c5-295b-4935-ae02-ed0c43f6faaf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:11.145159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3bc4a5c5-295b-4935-ae02-ed0c43f6faaf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:11.145613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3bc4a5c5-295b-4935-ae02-ed0c43f6faaf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:11.146360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3bc4a5c5-295b-4935-ae02-ed0c43f6faaf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:11.146783 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3bc4a5c5-295b-4935-ae02-ed0c43f6faaf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:11.164071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b30493b6-721a-41e5-9dbe-42da0e60a76d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:11.172465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b30493b6-721a-41e5-9dbe-42da0e60a76d tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.171699 May 25 16:44:11.173051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1200/2421] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:10 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:11.280754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3bc4a5c5-295b-4935-ae02-ed0c43f6faaf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:11.293692 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3bc4a5c5-295b-4935-ae02-ed0c43f6faaf tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.159591 May 25 16:44:11.294470 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1222/2422] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:11 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:44:11.655330 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:11.656256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:11.666936 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:11.667418 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:11.668173 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:11.668631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:11.675410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3a242720-de40-490b-9906-71459dc9eb62 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:11.676026 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3a242720-de40-490b-9906-71459dc9eb62 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:11.682993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3a242720-de40-490b-9906-71459dc9eb62 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:11.683481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3a242720-de40-490b-9906-71459dc9eb62 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:11.684246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3a242720-de40-490b-9906-71459dc9eb62 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:11.684683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3a242720-de40-490b-9906-71459dc9eb62 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:11.744875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:11.753667 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:11.755180 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:11.756335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-24761cbd-4c30-46d7-ad8b-efb9b4770e0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.107723 May 25 16:44:11.756883 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1201/2423] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:11 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:11.760462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-3a242720-de40-490b-9906-71459dc9eb62 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:11.768565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3a242720-de40-490b-9906-71459dc9eb62 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.097096 May 25 16:44:11.769151 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1223/2424] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:11 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:12.194441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-16753601-b4f3-464e-aeb2-a70d5f6e1da4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:12.195698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-16753601-b4f3-464e-aeb2-a70d5f6e1da4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:12.205927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-16753601-b4f3-464e-aeb2-a70d5f6e1da4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:12.206674 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-16753601-b4f3-464e-aeb2-a70d5f6e1da4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:12.207963 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-16753601-b4f3-464e-aeb2-a70d5f6e1da4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:12.208679 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-16753601-b4f3-464e-aeb2-a70d5f6e1da4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:12.315916 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-727a0653-a96b-4369-9988-7abde55db927 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:12.316533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-727a0653-a96b-4369-9988-7abde55db927 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:12.322792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-727a0653-a96b-4369-9988-7abde55db927 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:12.323342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-727a0653-a96b-4369-9988-7abde55db927 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:12.324062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-727a0653-a96b-4369-9988-7abde55db927 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:12.324500 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-727a0653-a96b-4369-9988-7abde55db927 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:12.343668 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-16753601-b4f3-464e-aeb2-a70d5f6e1da4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:12.351760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-16753601-b4f3-464e-aeb2-a70d5f6e1da4 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.164064 May 25 16:44:12.352324 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1202/2425] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:12 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:12.456555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-727a0653-a96b-4369-9988-7abde55db927 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:12.469134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-727a0653-a96b-4369-9988-7abde55db927 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.156860 May 25 16:44:12.469888 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1224/2426] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:12 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:12.774389 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:12.775011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:12.782089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:12.782535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:12.783310 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:12.783737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:12.784420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4f170029-3fc4-4fa0-84bf-f1ca46570111 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:12.785032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4f170029-3fc4-4fa0-84bf-f1ca46570111 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:12.792180 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4f170029-3fc4-4fa0-84bf-f1ca46570111 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:12.792634 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4f170029-3fc4-4fa0-84bf-f1ca46570111 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:12.793395 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4f170029-3fc4-4fa0-84bf-f1ca46570111 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:12.793861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4f170029-3fc4-4fa0-84bf-f1ca46570111 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:12.853187 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:12.861893 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:12.863420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:12.864558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5336a2e1-369b-49c8-a7ef-ae3308313d4b tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.094333 May 25 16:44:12.865113 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1203/2427] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:12 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:12.869463 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4f170029-3fc4-4fa0-84bf-f1ca46570111 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:12.877730 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4f170029-3fc4-4fa0-84bf-f1ca46570111 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.097188 May 25 16:44:12.878300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1225/2428] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:12 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:13.372585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-60f39422-0eb5-4276-b395-9267cb07be4b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:13.373468 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-60f39422-0eb5-4276-b395-9267cb07be4b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:13.383837 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60f39422-0eb5-4276-b395-9267cb07be4b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:13.384541 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60f39422-0eb5-4276-b395-9267cb07be4b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:13.385304 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60f39422-0eb5-4276-b395-9267cb07be4b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:13.385772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-60f39422-0eb5-4276-b395-9267cb07be4b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:13.489792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b36517e4-4107-4e1e-8034-35df2098c975 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:13.490409 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b36517e4-4107-4e1e-8034-35df2098c975 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:13.492491 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:13.498436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b36517e4-4107-4e1e-8034-35df2098c975 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:13.498857 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b36517e4-4107-4e1e-8034-35df2098c975 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:13.499627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b36517e4-4107-4e1e-8034-35df2098c975 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:13.500056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b36517e4-4107-4e1e-8034-35df2098c975 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:13.520501 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-60f39422-0eb5-4276-b395-9267cb07be4b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:13.529371 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-60f39422-0eb5-4276-b395-9267cb07be4b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.163323 May 25 16:44:13.529996 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1204/2429] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:13 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:13.656571 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b36517e4-4107-4e1e-8034-35df2098c975 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:13.669254 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b36517e4-4107-4e1e-8034-35df2098c975 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.183186 May 25 16:44:13.670098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1226/2430] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:13 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:13.886300 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:13.887323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:13.898471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:13.899303 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:13.900295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-62ccad27-2a36-4809-8aa4-e627199f808f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:13.901057 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:13.901239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:13.901403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-62ccad27-2a36-4809-8aa4-e627199f808f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:13.912343 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62ccad27-2a36-4809-8aa4-e627199f808f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:13.912982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62ccad27-2a36-4809-8aa4-e627199f808f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:13.914596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62ccad27-2a36-4809-8aa4-e627199f808f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:13.915384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-62ccad27-2a36-4809-8aa4-e627199f808f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:13.982419 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:13.991204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:13.992736 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:13.993873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5886dd57-d3f5-4c50-8870-615f426e48f5 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.114402 May 25 16:44:13.994412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1205/2431] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:13 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:14.000517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-62ccad27-2a36-4809-8aa4-e627199f808f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:14.010789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-62ccad27-2a36-4809-8aa4-e627199f808f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.119070 May 25 16:44:14.011374 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1227/2432] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:13 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:14.550955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-433cb05c-e3d3-4d56-84c4-546c896f4388 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:14.551977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-433cb05c-e3d3-4d56-84c4-546c896f4388 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:14.560077 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-433cb05c-e3d3-4d56-84c4-546c896f4388 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:14.560527 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-433cb05c-e3d3-4d56-84c4-546c896f4388 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:14.561273 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-433cb05c-e3d3-4d56-84c4-546c896f4388 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:14.561705 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-433cb05c-e3d3-4d56-84c4-546c896f4388 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:14.687130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-40588e5d-e248-43f5-bc21-ef3a2454be89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:14.687708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-40588e5d-e248-43f5-bc21-ef3a2454be89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:14.694359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40588e5d-e248-43f5-bc21-ef3a2454be89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:14.694828 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40588e5d-e248-43f5-bc21-ef3a2454be89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:14.695602 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40588e5d-e248-43f5-bc21-ef3a2454be89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:14.696038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-40588e5d-e248-43f5-bc21-ef3a2454be89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:14.696157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-433cb05c-e3d3-4d56-84c4-546c896f4388 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:14.704738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-433cb05c-e3d3-4d56-84c4-546c896f4388 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.160275 May 25 16:44:14.705320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1206/2433] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:14 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:14.832390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-40588e5d-e248-43f5-bc21-ef3a2454be89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:14.844979 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-40588e5d-e248-43f5-bc21-ef3a2454be89 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.161499 May 25 16:44:14.845769 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1228/2434] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:14 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:15.016205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:15.017082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:15.027366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:15.027821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:15.028571 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:15.029009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:15.031475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-34f6439b-968f-4f2a-8d09-53103d181836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:15.032087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-34f6439b-968f-4f2a-8d09-53103d181836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:15.040294 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34f6439b-968f-4f2a-8d09-53103d181836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:15.040718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34f6439b-968f-4f2a-8d09-53103d181836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:15.041490 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34f6439b-968f-4f2a-8d09-53103d181836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:15.041924 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-34f6439b-968f-4f2a-8d09-53103d181836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:15.101826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:15.110714 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:15.112252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:15.113393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aa066896-377e-4b32-bc96-ccff40cfc4ff tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.103822 May 25 16:44:15.113930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1207/2435] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:15 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:15.127565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-34f6439b-968f-4f2a-8d09-53103d181836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:15.135773 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-34f6439b-968f-4f2a-8d09-53103d181836 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.109705 May 25 16:44:15.136344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1229/2436] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:15 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:15.727608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-620d8f62-3828-477e-a274-4caadf5074e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:15.728550 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-620d8f62-3828-477e-a274-4caadf5074e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:15.735233 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-620d8f62-3828-477e-a274-4caadf5074e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:15.735617 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-620d8f62-3828-477e-a274-4caadf5074e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:15.736373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-620d8f62-3828-477e-a274-4caadf5074e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:15.736799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-620d8f62-3828-477e-a274-4caadf5074e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:15.861999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f6b16533-52ca-41df-887a-5b78da5f082a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:15.862627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f6b16533-52ca-41df-887a-5b78da5f082a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:15.869146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-620d8f62-3828-477e-a274-4caadf5074e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:15.869344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f6b16533-52ca-41df-887a-5b78da5f082a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:15.869813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f6b16533-52ca-41df-887a-5b78da5f082a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:15.870562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f6b16533-52ca-41df-887a-5b78da5f082a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:15.870999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f6b16533-52ca-41df-887a-5b78da5f082a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:15.877476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-620d8f62-3828-477e-a274-4caadf5074e3 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.157977 May 25 16:44:15.878239 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1208/2437] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:15 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:16.021006 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f6b16533-52ca-41df-887a-5b78da5f082a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:16.029343 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f6b16533-52ca-41df-887a-5b78da5f082a tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.171318 May 25 16:44:16.029951 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1230/2438] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:15 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:16.135319 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:16.135906 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:16.143195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:16.143678 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:16.144481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:16.144943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:16.153250 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-aa8669da-9d84-4781-93d3-a95fb6f96dcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:16.153895 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-aa8669da-9d84-4781-93d3-a95fb6f96dcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:16.161197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aa8669da-9d84-4781-93d3-a95fb6f96dcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:16.161671 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aa8669da-9d84-4781-93d3-a95fb6f96dcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:16.162512 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aa8669da-9d84-4781-93d3-a95fb6f96dcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:16.162954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aa8669da-9d84-4781-93d3-a95fb6f96dcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:16.218260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:16.227235 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:16.228861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:16.230001 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-aa1bb0d4-b6b3-48af-9c06-355e6da08316 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.100933 May 25 16:44:16.230597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1209/2439] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:16 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:16.234144 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aa8669da-9d84-4781-93d3-a95fb6f96dcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:16.242859 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-aa8669da-9d84-4781-93d3-a95fb6f96dcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093907 May 25 16:44:16.243452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1231/2440] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:16 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:16.900217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ac7cc060-c472-4e21-aa0a-4cd218c89ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:16.901185 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ac7cc060-c472-4e21-aa0a-4cd218c89ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:16.911409 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac7cc060-c472-4e21-aa0a-4cd218c89ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:16.912167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac7cc060-c472-4e21-aa0a-4cd218c89ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:16.913559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac7cc060-c472-4e21-aa0a-4cd218c89ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:16.914615 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ac7cc060-c472-4e21-aa0a-4cd218c89ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:17.046069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ac7cc060-c472-4e21-aa0a-4cd218c89ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:17.047489 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cdb980f5-d501-4678-bf10-0d0236b8cde3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:17.048186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cdb980f5-d501-4678-bf10-0d0236b8cde3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:17.054292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cdb980f5-d501-4678-bf10-0d0236b8cde3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:17.054464 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ac7cc060-c472-4e21-aa0a-4cd218c89ae7 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.160821 May 25 16:44:17.054747 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cdb980f5-d501-4678-bf10-0d0236b8cde3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:17.055059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1210/2441] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:16 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:17.055533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cdb980f5-d501-4678-bf10-0d0236b8cde3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:17.055972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cdb980f5-d501-4678-bf10-0d0236b8cde3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:17.189587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cdb980f5-d501-4678-bf10-0d0236b8cde3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:17.198252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cdb980f5-d501-4678-bf10-0d0236b8cde3 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.154858 May 25 16:44:17.198829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1232/2442] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:17 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:17.251846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:17.252755 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:17.263227 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:17.263919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:17.264990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:17.265608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:17.267488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6fbf86a8-c041-4bd4-8c65-577d2376acc2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:17.268355 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6fbf86a8-c041-4bd4-8c65-577d2376acc2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:17.276433 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fbf86a8-c041-4bd4-8c65-577d2376acc2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:17.276902 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fbf86a8-c041-4bd4-8c65-577d2376acc2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:17.277663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fbf86a8-c041-4bd4-8c65-577d2376acc2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:17.278085 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6fbf86a8-c041-4bd4-8c65-577d2376acc2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:17.338171 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:17.347062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:17.348588 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:17.349718 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d909d5f1-8567-4f53-93f3-080ab3c22c0f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.104553 May 25 16:44:17.350249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1211/2443] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:17 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:17.354831 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6fbf86a8-c041-4bd4-8c65-577d2376acc2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:17.363176 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6fbf86a8-c041-4bd4-8c65-577d2376acc2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.101733 May 25 16:44:17.363739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1233/2444] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:17 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:18.076065 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-03ea6bc5-c031-465e-ab53-8f95a5101e89 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:18.077082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-03ea6bc5-c031-465e-ab53-8f95a5101e89 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:18.087548 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-03ea6bc5-c031-465e-ab53-8f95a5101e89 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:18.088015 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-03ea6bc5-c031-465e-ab53-8f95a5101e89 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:18.088791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-03ea6bc5-c031-465e-ab53-8f95a5101e89 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:18.089210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-03ea6bc5-c031-465e-ab53-8f95a5101e89 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:18.104299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:18.216190 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0c43ba4c-9966-4dab-b431-521feecc2e1b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:18.216801 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0c43ba4c-9966-4dab-b431-521feecc2e1b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:18.221481 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-03ea6bc5-c031-465e-ab53-8f95a5101e89 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:18.222994 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c43ba4c-9966-4dab-b431-521feecc2e1b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:18.223494 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c43ba4c-9966-4dab-b431-521feecc2e1b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:18.224261 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c43ba4c-9966-4dab-b431-521feecc2e1b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:18.224743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c43ba4c-9966-4dab-b431-521feecc2e1b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:18.229706 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-03ea6bc5-c031-465e-ab53-8f95a5101e89 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.160110 May 25 16:44:18.230260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1212/2445] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:18 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:18.366253 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0c43ba4c-9966-4dab-b431-521feecc2e1b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:18.371412 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:18.372027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:18.376719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0c43ba4c-9966-4dab-b431-521feecc2e1b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.164660 May 25 16:44:18.377297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1234/2446] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:18 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:18.379120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:18.379619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:18.380387 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:18.380808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:18.381699 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ba8ffe66-e75f-44fa-a42d-31318c4ee2ce tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:18.382335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ba8ffe66-e75f-44fa-a42d-31318c4ee2ce tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:18.388683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ba8ffe66-e75f-44fa-a42d-31318c4ee2ce tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:18.389136 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ba8ffe66-e75f-44fa-a42d-31318c4ee2ce tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:18.389899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ba8ffe66-e75f-44fa-a42d-31318c4ee2ce tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:18.390337 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ba8ffe66-e75f-44fa-a42d-31318c4ee2ce tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:18.455903 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:18.464682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:18.467177 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:18.468590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b7a545bf-9272-4f67-8883-163b7589c4f1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.103080 May 25 16:44:18.469164 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1213/2447] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:18 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:18.472917 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ba8ffe66-e75f-44fa-a42d-31318c4ee2ce tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:18.486218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ba8ffe66-e75f-44fa-a42d-31318c4ee2ce tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.107972 May 25 16:44:18.487010 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1235/2448] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:18 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:18.802887 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:44:18.803267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:44:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:44:19.251052 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7a7365de-5675-42fd-8381-5f682ebe6390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:19.252101 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7a7365de-5675-42fd-8381-5f682ebe6390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:19.261328 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a7365de-5675-42fd-8381-5f682ebe6390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:19.261774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a7365de-5675-42fd-8381-5f682ebe6390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:19.262526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a7365de-5675-42fd-8381-5f682ebe6390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:19.262961 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7a7365de-5675-42fd-8381-5f682ebe6390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:19.393984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2ac14a8f-2601-4649-920a-1f3ba41a1d9e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:19.394621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2ac14a8f-2601-4649-920a-1f3ba41a1d9e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:19.402290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2ac14a8f-2601-4649-920a-1f3ba41a1d9e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:19.402473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7a7365de-5675-42fd-8381-5f682ebe6390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:19.402882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2ac14a8f-2601-4649-920a-1f3ba41a1d9e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:19.403532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2ac14a8f-2601-4649-920a-1f3ba41a1d9e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:19.403968 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2ac14a8f-2601-4649-920a-1f3ba41a1d9e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:19.410839 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7a7365de-5675-42fd-8381-5f682ebe6390 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.166193 May 25 16:44:19.411452 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1214/2449] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:19 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:19.486323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:19.486919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:19.493100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:19.493740 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:19.494529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:19.494950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:19.545370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-2ac14a8f-2601-4649-920a-1f3ba41a1d9e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:19.553591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2ac14a8f-2601-4649-920a-1f3ba41a1d9e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.163482 May 25 16:44:19.554146 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1236/2450] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:19 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:19.558638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-79248151-ff8a-4ef2-90a1-bb58aa5e9bc8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:19.559321 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-79248151-ff8a-4ef2-90a1-bb58aa5e9bc8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:19.561105 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:19.565547 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-79248151-ff8a-4ef2-90a1-bb58aa5e9bc8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:19.566001 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-79248151-ff8a-4ef2-90a1-bb58aa5e9bc8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:19.566766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-79248151-ff8a-4ef2-90a1-bb58aa5e9bc8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:19.567235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-79248151-ff8a-4ef2-90a1-bb58aa5e9bc8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:19.569938 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:19.571475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:19.572628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-7aa9c632-d3c3-449c-a60c-fe17f55cfaa6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.090201 May 25 16:44:19.573172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1215/2451] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:19 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:19.627761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-79248151-ff8a-4ef2-90a1-bb58aa5e9bc8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:19.636213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-79248151-ff8a-4ef2-90a1-bb58aa5e9bc8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.081107 May 25 16:44:19.636789 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1237/2452] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:19 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:20.432170 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-6e86f85d-ee2e-478d-b55b-db73e1237192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:20.432971 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-6e86f85d-ee2e-478d-b55b-db73e1237192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:20.440828 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e86f85d-ee2e-478d-b55b-db73e1237192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:20.441542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e86f85d-ee2e-478d-b55b-db73e1237192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:20.443427 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e86f85d-ee2e-478d-b55b-db73e1237192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:20.445567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-6e86f85d-ee2e-478d-b55b-db73e1237192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:20.571188 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a52d860a-66c2-424d-ae1a-5372ab78a1d4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:20.571794 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a52d860a-66c2-424d-ae1a-5372ab78a1d4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:20.578485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a52d860a-66c2-424d-ae1a-5372ab78a1d4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:20.578665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-6e86f85d-ee2e-478d-b55b-db73e1237192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:20.578933 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a52d860a-66c2-424d-ae1a-5372ab78a1d4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:20.579733 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a52d860a-66c2-424d-ae1a-5372ab78a1d4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:20.580166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a52d860a-66c2-424d-ae1a-5372ab78a1d4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:20.588580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-6e86f85d-ee2e-478d-b55b-db73e1237192 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.163067 May 25 16:44:20.589188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1216/2453] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:20 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:20.593800 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:20.594383 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:20.596539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:20.600838 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:20.601251 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:20.602025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:20.602448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:20.918206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:20.927217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:20.928738 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:20.930665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-28b32f54-c6dc-47b8-90f8-fa492074b6e6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.340523 May 25 16:44:20.931477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1217/2454] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:20 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:20.935614 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-343d6615-4e7d-40fb-a36d-f2654946bfcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:20.936220 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-343d6615-4e7d-40fb-a36d-f2654946bfcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:20.942682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-343d6615-4e7d-40fb-a36d-f2654946bfcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:20.943138 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-343d6615-4e7d-40fb-a36d-f2654946bfcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:20.943960 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-343d6615-4e7d-40fb-a36d-f2654946bfcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:20.944605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-343d6615-4e7d-40fb-a36d-f2654946bfcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:20.976196 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-a52d860a-66c2-424d-ae1a-5372ab78a1d4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:20.984842 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a52d860a-66c2-424d-ae1a-5372ab78a1d4 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.417598 May 25 16:44:20.986276 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1238/2455] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:20 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:21.005166 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-343d6615-4e7d-40fb-a36d-f2654946bfcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:21.015034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-343d6615-4e7d-40fb-a36d-f2654946bfcd tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.082525 May 25 16:44:21.015862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1218/2456] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:20 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:21.609382 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0c94727a-5b5e-4981-acd5-bd4f27f74a90 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:21.610025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0c94727a-5b5e-4981-acd5-bd4f27f74a90 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:21.620814 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c94727a-5b5e-4981-acd5-bd4f27f74a90 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:21.621419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c94727a-5b5e-4981-acd5-bd4f27f74a90 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:21.622514 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c94727a-5b5e-4981-acd5-bd4f27f74a90 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:21.623113 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c94727a-5b5e-4981-acd5-bd4f27f74a90 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:21.761128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0c94727a-5b5e-4981-acd5-bd4f27f74a90 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:21.769462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0c94727a-5b5e-4981-acd5-bd4f27f74a90 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.165669 May 25 16:44:21.770034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1239/2457] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:21 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:21.955486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:21.956379 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:21.969406 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:21.969891 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:21.970645 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:21.971074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:22.002333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-28683fc7-6026-4bd5-b3a5-34618aaf0c5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:22.003048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-28683fc7-6026-4bd5-b3a5-34618aaf0c5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:22.009660 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-28683fc7-6026-4bd5-b3a5-34618aaf0c5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:22.010198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-28683fc7-6026-4bd5-b3a5-34618aaf0c5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:22.011012 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-28683fc7-6026-4bd5-b3a5-34618aaf0c5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:22.011569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-28683fc7-6026-4bd5-b3a5-34618aaf0c5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:22.034258 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:22.043340 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:22.044909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:22.046083 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-951066a5-e67f-4c1e-ac4c-ee2f253eb363 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098424 May 25 16:44:22.046642 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1219/2458] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:21 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:22.050948 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-303e6967-9e59-420a-8d2d-7fe821d79e22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:22.051682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-303e6967-9e59-420a-8d2d-7fe821d79e22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:22.058918 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-303e6967-9e59-420a-8d2d-7fe821d79e22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:22.059401 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-303e6967-9e59-420a-8d2d-7fe821d79e22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:22.060153 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-303e6967-9e59-420a-8d2d-7fe821d79e22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:22.060580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-303e6967-9e59-420a-8d2d-7fe821d79e22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:22.135291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-303e6967-9e59-420a-8d2d-7fe821d79e22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:22.143699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-303e6967-9e59-420a-8d2d-7fe821d79e22 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.096088 May 25 16:44:22.144626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1220/2459] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:22 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:22.165511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-28683fc7-6026-4bd5-b3a5-34618aaf0c5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:22.174008 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-28683fc7-6026-4bd5-b3a5-34618aaf0c5b tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.175623 May 25 16:44:22.174583 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1240/2460] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:21 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:22.791594 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c086d6c0-b092-4d35-8469-96692b357277 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:22.792572 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c086d6c0-b092-4d35-8469-96692b357277 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:22.803050 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c086d6c0-b092-4d35-8469-96692b357277 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:22.803809 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c086d6c0-b092-4d35-8469-96692b357277 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:22.805021 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c086d6c0-b092-4d35-8469-96692b357277 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:22.805770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c086d6c0-b092-4d35-8469-96692b357277 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:22.942521 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c086d6c0-b092-4d35-8469-96692b357277 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:22.953505 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c086d6c0-b092-4d35-8469-96692b357277 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.169017 May 25 16:44:22.954076 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1221/2461] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:22 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:23.068336 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:23.069280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:23.077453 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:23.077905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:23.078680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:23.079120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:23.140485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:23.149319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:23.150981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:23.152184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-aea800f2-c3f1-4f4c-ba3c-240abbe772b4 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.090615 May 25 16:44:23.152775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1241/2462] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:23 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:23.161285 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c2086f60-2eb7-4e47-a02c-464e1703314c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:23.161915 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c2086f60-2eb7-4e47-a02c-464e1703314c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:23.169212 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c2086f60-2eb7-4e47-a02c-464e1703314c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:23.169666 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c2086f60-2eb7-4e47-a02c-464e1703314c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:23.170423 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c2086f60-2eb7-4e47-a02c-464e1703314c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:23.170858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c2086f60-2eb7-4e47-a02c-464e1703314c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:23.191065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b2952a0f-2b81-4898-8bd7-4737b70b2672 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:23.191705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b2952a0f-2b81-4898-8bd7-4737b70b2672 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:23.198358 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2952a0f-2b81-4898-8bd7-4737b70b2672 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:23.198809 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2952a0f-2b81-4898-8bd7-4737b70b2672 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:23.199641 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2952a0f-2b81-4898-8bd7-4737b70b2672 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:23.200077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b2952a0f-2b81-4898-8bd7-4737b70b2672 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:23.233676 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c2086f60-2eb7-4e47-a02c-464e1703314c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:23.242071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c2086f60-2eb7-4e47-a02c-464e1703314c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084626 May 25 16:44:23.242675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1222/2463] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:23 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:23.344536 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b2952a0f-2b81-4898-8bd7-4737b70b2672 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:23.353120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b2952a0f-2b81-4898-8bd7-4737b70b2672 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.165860 May 25 16:44:23.353696 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1242/2464] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:23 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:23.457304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:44:23.457656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:44:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:44:23.975766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3289114f-78dd-4ada-83f9-b2f06aadce37 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:23.976730 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3289114f-78dd-4ada-83f9-b2f06aadce37 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:23.985320 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3289114f-78dd-4ada-83f9-b2f06aadce37 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:23.985871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3289114f-78dd-4ada-83f9-b2f06aadce37 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:23.987122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3289114f-78dd-4ada-83f9-b2f06aadce37 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:23.987580 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3289114f-78dd-4ada-83f9-b2f06aadce37 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:24.123697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3289114f-78dd-4ada-83f9-b2f06aadce37 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:24.136532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3289114f-78dd-4ada-83f9-b2f06aadce37 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.167556 May 25 16:44:24.137367 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1223/2465] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:23 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:24.170296 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:24.171049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:24.178246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:24.178700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:24.179499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:24.179944 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:24.252784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:24.261470 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bf860bf3-36ef-4a1b-8709-fd9a747e3264 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:24.262049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-bf860bf3-36ef-4a1b-8709-fd9a747e3264 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:24.265339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:24.267418 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:24.269168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-73144c6d-2f8b-4ffa-bdcd-b1643478b242 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.102532 May 25 16:44:24.269878 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1243/2466] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:24 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:24.271196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf860bf3-36ef-4a1b-8709-fd9a747e3264 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:24.271715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf860bf3-36ef-4a1b-8709-fd9a747e3264 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:24.272628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf860bf3-36ef-4a1b-8709-fd9a747e3264 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:24.273058 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-bf860bf3-36ef-4a1b-8709-fd9a747e3264 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:24.337469 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-bf860bf3-36ef-4a1b-8709-fd9a747e3264 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:24.348807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bf860bf3-36ef-4a1b-8709-fd9a747e3264 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.093148 May 25 16:44:24.349602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1224/2467] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:24 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:24.370281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-cc27682e-9d78-42eb-985f-b5e47d7b3e4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:24.370890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-cc27682e-9d78-42eb-985f-b5e47d7b3e4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:24.377169 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc27682e-9d78-42eb-985f-b5e47d7b3e4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:24.377617 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc27682e-9d78-42eb-985f-b5e47d7b3e4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:24.378365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc27682e-9d78-42eb-985f-b5e47d7b3e4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:24.378794 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-cc27682e-9d78-42eb-985f-b5e47d7b3e4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:24.523763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-cc27682e-9d78-42eb-985f-b5e47d7b3e4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:24.532342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-cc27682e-9d78-42eb-985f-b5e47d7b3e4e tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.165963 May 25 16:44:24.532925 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1244/2468] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:24 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:25.160375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fc62766f-992f-4fa3-9bf4-d54637f0f960 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:25.160981 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-fc62766f-992f-4fa3-9bf4-d54637f0f960 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:25.169695 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc62766f-992f-4fa3-9bf4-d54637f0f960 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:25.170154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc62766f-992f-4fa3-9bf4-d54637f0f960 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:25.170899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc62766f-992f-4fa3-9bf4-d54637f0f960 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:25.171397 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-fc62766f-992f-4fa3-9bf4-d54637f0f960 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:25.287732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:25.288428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:25.295700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:25.296137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:25.296891 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:25.297318 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:25.308742 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-fc62766f-992f-4fa3-9bf4-d54637f0f960 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:25.317167 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fc62766f-992f-4fa3-9bf4-d54637f0f960 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.160688 May 25 16:44:25.317711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1225/2469] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:25 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:25.358340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:25.365819 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ccc33e93-9751-458e-84ba-e2e054a87afb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:25.366421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ccc33e93-9751-458e-84ba-e2e054a87afb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:25.367353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:25.368860 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:25.369993 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-49d644bf-9192-458c-b931-a399e863a21d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.086338 May 25 16:44:25.370555 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1245/2470] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:25 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:25.372806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccc33e93-9751-458e-84ba-e2e054a87afb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:25.373249 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccc33e93-9751-458e-84ba-e2e054a87afb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:25.374000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccc33e93-9751-458e-84ba-e2e054a87afb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:25.374432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccc33e93-9751-458e-84ba-e2e054a87afb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:25.435022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ccc33e93-9751-458e-84ba-e2e054a87afb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:25.443038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ccc33e93-9751-458e-84ba-e2e054a87afb tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.081024 May 25 16:44:25.443629 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1226/2471] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:25 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:44:25.554326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b5bdc204-d1cf-413b-9a36-6027f78c3273 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:25.555402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b5bdc204-d1cf-413b-9a36-6027f78c3273 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:25.562508 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5bdc204-d1cf-413b-9a36-6027f78c3273 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:25.563050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5bdc204-d1cf-413b-9a36-6027f78c3273 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:25.563890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5bdc204-d1cf-413b-9a36-6027f78c3273 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:25.564351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b5bdc204-d1cf-413b-9a36-6027f78c3273 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:25.701123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b5bdc204-d1cf-413b-9a36-6027f78c3273 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:25.709598 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b5bdc204-d1cf-413b-9a36-6027f78c3273 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.161604 May 25 16:44:25.710171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1246/2472] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:25 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:26.338157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ed5c2cd0-e815-42c8-9521-f16d7149a97b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:26.338732 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ed5c2cd0-e815-42c8-9521-f16d7149a97b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:26.345559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed5c2cd0-e815-42c8-9521-f16d7149a97b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:26.346044 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed5c2cd0-e815-42c8-9521-f16d7149a97b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:26.346844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed5c2cd0-e815-42c8-9521-f16d7149a97b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:26.347332 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ed5c2cd0-e815-42c8-9521-f16d7149a97b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:26.387875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:26.388533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:26.395492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:26.395937 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:26.396697 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:26.397128 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:26.470972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:26.479885 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:26.481386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:26.482530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-39949b23-22a8-4a0d-9ce7-0324dc552b74 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.098637 May 25 16:44:26.483096 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1247/2473] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:26 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:26.487422 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-22a7673a-27f3-48b6-a018-375c78e7ba92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:26.488022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-22a7673a-27f3-48b6-a018-375c78e7ba92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:26.494304 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22a7673a-27f3-48b6-a018-375c78e7ba92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:26.494745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22a7673a-27f3-48b6-a018-375c78e7ba92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:26.495533 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22a7673a-27f3-48b6-a018-375c78e7ba92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:26.495967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-22a7673a-27f3-48b6-a018-375c78e7ba92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:26.509877 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ed5c2cd0-e815-42c8-9521-f16d7149a97b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:26.518382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ed5c2cd0-e815-42c8-9521-f16d7149a97b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.186450 May 25 16:44:26.518988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1227/2474] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:26 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:26.556836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-22a7673a-27f3-48b6-a018-375c78e7ba92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:26.564972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-22a7673a-27f3-48b6-a018-375c78e7ba92 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.080934 May 25 16:44:26.565521 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1248/2475] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:26 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:26.730658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-58c6a541-938b-4789-919a-546aa656ba07 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:26.731384 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-58c6a541-938b-4789-919a-546aa656ba07 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:26.739305 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58c6a541-938b-4789-919a-546aa656ba07 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:26.739904 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58c6a541-938b-4789-919a-546aa656ba07 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:26.740840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58c6a541-938b-4789-919a-546aa656ba07 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:26.741388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-58c6a541-938b-4789-919a-546aa656ba07 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:26.881014 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-58c6a541-938b-4789-919a-546aa656ba07 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:26.889434 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-58c6a541-938b-4789-919a-546aa656ba07 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.164826 May 25 16:44:26.890006 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1228/2476] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:26 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:27.505161 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:27.506110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:27.514365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:27.514811 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:27.515593 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:27.516022 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:27.537228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a9119de0-bf27-4385-9b13-af6b7592bdb6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:27.537861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a9119de0-bf27-4385-9b13-af6b7592bdb6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:27.547368 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9119de0-bf27-4385-9b13-af6b7592bdb6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:27.547815 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9119de0-bf27-4385-9b13-af6b7592bdb6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:27.548582 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9119de0-bf27-4385-9b13-af6b7592bdb6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:27.549008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a9119de0-bf27-4385-9b13-af6b7592bdb6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:27.551754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:27.589370 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:27.598231 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:27.599755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:27.601032 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ebad4db2-f91f-455d-be31-42f7400f8815 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.103624 May 25 16:44:27.601587 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1249/2477] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:27 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:27.611245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-91dafb6a-27b3-471b-bd14-5c09de10acaa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:27.611877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-91dafb6a-27b3-471b-bd14-5c09de10acaa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:27.618057 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-91dafb6a-27b3-471b-bd14-5c09de10acaa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:27.618565 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-91dafb6a-27b3-471b-bd14-5c09de10acaa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:27.619277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-91dafb6a-27b3-471b-bd14-5c09de10acaa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:27.619707 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-91dafb6a-27b3-471b-bd14-5c09de10acaa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:27.688931 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-91dafb6a-27b3-471b-bd14-5c09de10acaa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:27.697087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-91dafb6a-27b3-471b-bd14-5c09de10acaa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.094542 May 25 16:44:27.697655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1250/2478] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:27 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:27.715890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-a9119de0-bf27-4385-9b13-af6b7592bdb6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:27.728515 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a9119de0-bf27-4385-9b13-af6b7592bdb6 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.195215 May 25 16:44:27.729073 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1229/2479] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:27 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:27.910745 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-66cc3f31-6456-4f7f-bc63-70510e09d918 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:27.911452 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-66cc3f31-6456-4f7f-bc63-70510e09d918 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:27.920297 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-66cc3f31-6456-4f7f-bc63-70510e09d918 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:27.920843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-66cc3f31-6456-4f7f-bc63-70510e09d918 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:27.921676 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-66cc3f31-6456-4f7f-bc63-70510e09d918 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:27.922251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-66cc3f31-6456-4f7f-bc63-70510e09d918 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:28.056198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-66cc3f31-6456-4f7f-bc63-70510e09d918 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:28.066201 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-66cc3f31-6456-4f7f-bc63-70510e09d918 tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.163356 May 25 16:44:28.066774 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1251/2480] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:27 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:28.622388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:28.623087 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:28.630561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:28.631089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:28.632019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:28.632525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:28.698410 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:28.707084 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:28.710096 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:28.711295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cb3b6683-e86a-4d08-875d-99bd7f207158 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.094497 May 25 16:44:28.711955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1230/2481] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:28 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:28.714625 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-df614138-58c4-4c40-871e-abe6e91d084d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:28.715295 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-df614138-58c4-4c40-871e-abe6e91d084d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:28.722240 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df614138-58c4-4c40-871e-abe6e91d084d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:28.722721 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df614138-58c4-4c40-871e-abe6e91d084d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:28.723589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df614138-58c4-4c40-871e-abe6e91d084d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:28.724023 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-df614138-58c4-4c40-871e-abe6e91d084d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:28.745748 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-33c0ac56-0dda-4e58-a4fb-53503a210ca8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:28.746359 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-33c0ac56-0dda-4e58-a4fb-53503a210ca8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:28.752799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-33c0ac56-0dda-4e58-a4fb-53503a210ca8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:28.753252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-33c0ac56-0dda-4e58-a4fb-53503a210ca8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:28.754042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-33c0ac56-0dda-4e58-a4fb-53503a210ca8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:28.754476 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-33c0ac56-0dda-4e58-a4fb-53503a210ca8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:28.784899 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-df614138-58c4-4c40-871e-abe6e91d084d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:28.794328 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-df614138-58c4-4c40-871e-abe6e91d084d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.084332 May 25 16:44:28.794915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1252/2482] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:28 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:44:28.892700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-33c0ac56-0dda-4e58-a4fb-53503a210ca8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:28.900964 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-33c0ac56-0dda-4e58-a4fb-53503a210ca8 tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.159150 May 25 16:44:28.901538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1231/2483] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:28 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:29.084519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d080980b-9dd1-4639-8339-41c9fe0a04aa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:29.085282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d080980b-9dd1-4639-8339-41c9fe0a04aa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:29.093175 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d080980b-9dd1-4639-8339-41c9fe0a04aa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:29.093740 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d080980b-9dd1-4639-8339-41c9fe0a04aa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:29.094753 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d080980b-9dd1-4639-8339-41c9fe0a04aa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:29.095343 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d080980b-9dd1-4639-8339-41c9fe0a04aa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:29.233143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d080980b-9dd1-4639-8339-41c9fe0a04aa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:29.245049 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d080980b-9dd1-4639-8339-41c9fe0a04aa tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.165360 May 25 16:44:29.245619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1253/2484] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:29 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:29.731205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:29.731882 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:29.738558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:29.739008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:29.739768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:29.740212 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:29.812524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b7d4830a-9e17-46cb-a6d2-72d5bcb3abf2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:29.813140 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b7d4830a-9e17-46cb-a6d2-72d5bcb3abf2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:29.822280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b7d4830a-9e17-46cb-a6d2-72d5bcb3abf2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:29.822754 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b7d4830a-9e17-46cb-a6d2-72d5bcb3abf2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:29.823531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b7d4830a-9e17-46cb-a6d2-72d5bcb3abf2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:29.823957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b7d4830a-9e17-46cb-a6d2-72d5bcb3abf2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:29.834390 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:29.844543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:29.846019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:29.847430 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dca2e424-bc47-4550-a24a-3000fcc10256 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.120601 May 25 16:44:29.848525 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1232/2485] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:29 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:29.896078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b7d4830a-9e17-46cb-a6d2-72d5bcb3abf2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:29.904311 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b7d4830a-9e17-46cb-a6d2-72d5bcb3abf2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.097570 May 25 16:44:29.904889 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1254/2486] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:29 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:29.918764 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ccb08f64-3235-4f35-b68d-144355d3089b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:29.919409 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ccb08f64-3235-4f35-b68d-144355d3089b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] [instance: d86354dc-f3d4-409d-b331-9f708947be2a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:29.926545 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccb08f64-3235-4f35-b68d-144355d3089b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:29.927186 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccb08f64-3235-4f35-b68d-144355d3089b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:29.927986 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccb08f64-3235-4f35-b68d-144355d3089b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:29.928417 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ccb08f64-3235-4f35-b68d-144355d3089b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:30.149665 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ccb08f64-3235-4f35-b68d-144355d3089b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e40faab6a34f4d368462cdcafb7bcfea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '711b13e4ea3e4269a0082811f6708d87', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:30.159081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ccb08f64-3235-4f35-b68d-144355d3089b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a" status: 200 len: 1599 microversion: 2.1 time: 0.244164 May 25 16:44:30.159702 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1233/2487] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:29 2022] GET /compute/v2.1/servers/d86354dc-f3d4-409d-b331-9f708947be2a => generated 1599 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:30.176903 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-66c59b54-b21e-47c4-9693-cb3de0f8808b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] Calling method '.version_select at 0x7f4a301a2040>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:30.204063 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-66c59b54-b21e-47c4-9693-cb3de0f8808b tempest-TestMinimumBasicScenario-1349570600 tempest-TestMinimumBasicScenario-1349570600-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-151156222" status: 202 len: 0 microversion: 2.1 time: 0.030987 May 25 16:44:30.208148 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1255/2488] 10.222.0.45 () {58 vars in 1268 bytes} [Wed May 25 16:44:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-151156222 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:44:30.269373 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d480c864-0166-403f-8c69-4e90da60488f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:30.269996 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d480c864-0166-403f-8c69-4e90da60488f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] [instance: 3fc5dcaa-3275-47eb-88ec-62b23505c91b] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:30.276193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d480c864-0166-403f-8c69-4e90da60488f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:30.276710 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d480c864-0166-403f-8c69-4e90da60488f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:30.277394 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d480c864-0166-403f-8c69-4e90da60488f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:30.278473 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d480c864-0166-403f-8c69-4e90da60488f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:30.494024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-d480c864-0166-403f-8c69-4e90da60488f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b53b45a485b14e0b9273c019dc9ec206', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '073368e447af465abb41aa556ee46dc9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:30.505040 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d480c864-0166-403f-8c69-4e90da60488f tempest-VolumesSnapshotTestJSON-1046884794 tempest-VolumesSnapshotTestJSON-1046884794-project] 10.222.0.45 "GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b" status: 200 len: 1558 microversion: 2.1 time: 0.240904 May 25 16:44:30.506405 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1234/2489] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:30 2022] GET /compute/v2.1/servers/3fc5dcaa-3275-47eb-88ec-62b23505c91b => generated 1558 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:30.866043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:30.866646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:30.873566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:30.874009 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:30.874755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:30.875189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:30.877726 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=100 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:30.921498 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-46fd31b7-6f3a-4bb1-8bbe-17f5ce5a35a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:30.922069 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-46fd31b7-6f3a-4bb1-8bbe-17f5ce5a35a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:30.931491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46fd31b7-6f3a-4bb1-8bbe-17f5ce5a35a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:30.931491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46fd31b7-6f3a-4bb1-8bbe-17f5ce5a35a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:30.931491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46fd31b7-6f3a-4bb1-8bbe-17f5ce5a35a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:30.931491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-46fd31b7-6f3a-4bb1-8bbe-17f5ce5a35a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:30.941763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:30.950790 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:30.952366 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:30.953517 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-77793d3a-b254-450e-a25c-13ce98805155 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 200 len: 1444 microversion: 2.20 time: 0.091525 May 25 16:44:30.954077 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1256/2490] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:30 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:30.972688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-019a0f56-cc4c-4b05-adbe-2ee8507ebd50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '.version_select at 0x7f4a3004e4c0>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:30.993492 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-019a0f56-cc4c-4b05-adbe-2ee8507ebd50 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1711374587" status: 204 len: 0 microversion: 2.20 time: 0.024667 May 25 16:44:30.994480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1257/2491] 10.222.0.45 () {60 vars in 1261 bytes} [Wed May 25 16:44:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1711374587 => generated 0 bytes in 26 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 25 16:44:30.999945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-46fd31b7-6f3a-4bb1-8bbe-17f5ce5a35a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:31.013089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-46fd31b7-6f3a-4bb1-8bbe-17f5ce5a35a3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 200 len: 1372 microversion: 2.1 time: 0.095963 May 25 16:44:31.014125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1235/2492] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:30 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:31.030685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4f54ee1f-c9f7-4675-9934-76d367253e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '.version_select at 0x7f4a23fbb040>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:31.049883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4f54ee1f-c9f7-4675-9934-76d367253e3f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2106604150" status: 202 len: 0 microversion: 2.1 time: 0.023039 May 25 16:44:31.053905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1258/2493] 10.222.0.45 () {58 vars in 1220 bytes} [Wed May 25 16:44:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2106604150 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:44:31.900579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a0720ecb-f0ed-4ed3-a734-c901fa8a5fca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: .version_select at 0x7f4a30083280>, body: {"keypair": {"name": "tempest-keypair-1636892488", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJvXsj/Su8f1sZOh3Ceu6dvnE1EbMVAjPyzuOhc6pQyV2aeWOVh7vqCNIzypzkVLgY+f7c5AHlLkF7/RpP/c+p29+4fkRuzNT+FF0xjG/AP+gpTeGul0w+95Hlp4BrTsRw=="}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:31.917791 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-a0720ecb-f0ed-4ed3-a734-c901fa8a5fca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:44:31.923209 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-a0720ecb-f0ed-4ed3-a734-c901fa8a5fca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for user a0c9f3f10fd142439521e4b36cf33157 and project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:44:31.950085 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a0720ecb-f0ed-4ed3-a734-c901fa8a5fca tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.053372 May 25 16:44:31.950691 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1236/2494] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:44:31 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:44:32.241743 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3c95f97c-75a5-4eeb-8dd4-4b585a41f3ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Action: 'create', calling method: .version_select at 0x7f4a351aeb80>, body: {"keypair": {"name": "tempest-keypair-1949390145", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEFRJdS4A/IItHCLUr0SrP5OBK6BddnmjClm9fr8C1vrzlbLzr3Am51uj87GI1+pcVxADF7xXGCyyq9kEGEXhkVBW7izAe6M3NCrHh7CU8NyY6hOacNQl+HAqYv+uS+8lw=="}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:32.262271 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-3c95f97c-75a5-4eeb-8dd4-4b585a41f3ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Getting quotas for project c3003077416c4ce997e3aa7e47e8e3b7. Resources: {'key_pairs'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:44:32.267289 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-3c95f97c-75a5-4eeb-8dd4-4b585a41f3ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Getting quotas for user 06ba885a0b164134898c42a332feba76 and project c3003077416c4ce997e3aa7e47e8e3b7. Resources: {'key_pairs'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:44:32.298906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3c95f97c-75a5-4eeb-8dd4-4b585a41f3ac tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.062055 May 25 16:44:32.305912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1259/2495] 10.222.0.45 () {62 vars in 1199 bytes} [Wed May 25 16:44:32 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 70 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) May 25 16:44:34.365875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-655370530", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d4194a59-8440-4e29-8e43-bfe2dcadce67"}], "security_groups": [{"name": "tempest-securitygroup--1471938896"}], "key_name": "tempest-keypair-1949390145", "user_data": "}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:34.511100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.network.neutron [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] validate_networks() for [('d4194a59-8440-4e29-8e43-bfe2dcadce67', None, None, None, None, None)] {{(pid=117794) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:44:34.519527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1510571824", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "aaf08a79-3442-476f-b613-a02c12c36d8d"}], "security_groups": [{"name": "tempest-securitygroup--1827725928"}], "key_name": "tempest-keypair-1636892488", "user_data": "}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:34.685319 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] validate_networks() for [('aaf08a79-3442-476f-b613-a02c12c36d8d', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:44:35.313820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Flavor limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:44:35.314485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Image limits 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:44:35.315366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Flavor pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:44:35.316106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Image pref 0:0:0 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:44:35.316654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.virt.hardware [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117794) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:44:35.326587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:35.327093 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:35.328027 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:35.328544 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:35.339767 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:35.351293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Getting quotas for project c3003077416c4ce997e3aa7e47e8e3b7. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:44:35.355632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Getting quotas for user 06ba885a0b164134898c42a332feba76 and project c3003077416c4ce997e3aa7e47e8e3b7. Resources: {'ram', 'cores', 'instances'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:44:35.367585 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Going to run 1 instances... {{(pid=117794) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:44:35.369930 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117794) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:44:35.408627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:44:35.411110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:44:35.411658 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:44:35.412076 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:44:35.412515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:44:35.421517 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-506726e7-4417-439f-b1c2-4c1f4d82a2cc tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.060644 May 25 16:44:35.422799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1237/2496] 10.222.0.45 () {62 vars in 1187 bytes} [Wed May 25 16:44:34 2022] POST /compute/v2.1/servers => generated 402 bytes in 1062 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) May 25 16:44:35.427949 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:35.428524 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:35.432530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:35.432980 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:35.433778 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:35.434467 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:35.435844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:35.436299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:35.437199 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:35.437649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:35.451636 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] HTTP exception thrown: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.452550 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Request method failure captured: May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a HTTP/1.1 May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaSt3NdfdTff4OH_doadoVd4ihuLN8fcTpnktVD_XWHHevyVxY4tmWr46X1fVz2TuvDT5fSQ1evXFm6-DfkMDQmPYTRLtjUiNx8nVTeCIRoDB7T5LAC8qRTpjGqBOay7ujWQTOdtPe6UQNGpwsHetNPMu5dXLWOI1Kk0Q9IbllFii8 May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:44:35.453833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeTestJSON-755332015 May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeTestJSON-755332015-project May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: e4e4d39c5ed249bc9dc097ba4c24b3d2 May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeTestJSON-755332015-project May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.455689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'c98878d3-b9b6-4688-abea-b14c2150fb4a'} May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:44:35.456681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.458777 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.460853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Returning 404 to user: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:44:35.460853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a3f44a75-f245-4056-b650-22147f458a09 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "DELETE /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 404 len: 111 microversion: 2.1 time: 0.031192 May 25 16:44:35.460853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1238/2497] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:44:35 2022] DELETE /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:44:35.461097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for user a0c9f3f10fd142439521e4b36cf33157 and project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:44:35.461184 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4508845d-1457-4f03-a27d-e752d434609d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:35.461385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4508845d-1457-4f03-a27d-e752d434609d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:35.473798 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:44:35.476807 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:44:35.498213 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=3,nova_api:INSERT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:35.526465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4508845d-1457-4f03-a27d-e752d434609d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:35.536437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4508845d-1457-4f03-a27d-e752d434609d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:35.538342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-4508845d-1457-4f03-a27d-e752d434609d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:35.539989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4508845d-1457-4f03-a27d-e752d434609d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1281 microversion: 2.20 time: 0.083290 May 25 16:44:35.550497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1239/2498] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:35 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1281 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:35.554674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-58c33a99-5237-4be5-a469-aa188ee2736c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.036059 May 25 16:44:35.560718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1260/2499] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:44:34 2022] POST /compute/v2.1/servers => generated 402 bytes in 1046 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:44:35.565059 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:35.565873 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: c98878d3-b9b6-4688-abea-b14c2150fb4a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:35.568197 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8a816c32-811c-4e2d-938e-2ea00f21c04c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:35.568822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-8a816c32-811c-4e2d-938e-2ea00f21c04c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:35.577533 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:35.578338 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:35.580146 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:35.581038 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:35.596858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] HTTP exception thrown: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Request method failure captured: May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a HTTP/1.1 May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaSt3NdfdTff4OH_doadoVd4ihuLN8fcTpnktVD_XWHHevyVxY4tmWr46X1fVz2TuvDT5fSQ1evXFm6-DfkMDQmPYTRLtjUiNx8nVTeCIRoDB7T5LAC8qRTpjGqBOay7ujWQTOdtPe6UQNGpwsHetNPMu5dXLWOI1Kk0Q9IbllFii8 May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:44:35.598335 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeTestJSON-755332015 May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeTestJSON-755332015-project May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: e4e4d39c5ed249bc9dc097ba4c24b3d2 May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeTestJSON-755332015-project May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.600125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'c98878d3-b9b6-4688-abea-b14c2150fb4a'} May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:44:35.604554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. May 25 16:44:35.606217 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:44:35.607701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Returning 404 to user: Instance c98878d3-b9b6-4688-abea-b14c2150fb4a could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:44:35.607701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-80cd063d-5bba-4af7-8a3d-c94e225d3bb8 tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a" status: 404 len: 111 microversion: 2.1 time: 0.047626 May 25 16:44:35.607701 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1240/2500] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:35 2022] GET /compute/v2.1/servers/c98878d3-b9b6-4688-abea-b14c2150fb4a => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:44:35.645825 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-8a816c32-811c-4e2d-938e-2ea00f21c04c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:35.654939 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8a816c32-811c-4e2d-938e-2ea00f21c04c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1242 microversion: 2.1 time: 0.092813 May 25 16:44:35.655538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1261/2501] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:35 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1242 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:36.558350 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:36.559054 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:36.566034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:36.566567 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:36.567441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:36.567966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:36.651509 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:36.660213 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:36.661824 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:36.663193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e2c9afeb-1ecb-4bff-b2f8-c04b3d0d425c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1333 microversion: 2.20 time: 0.109132 May 25 16:44:36.664331 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1241/2502] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:36 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1333 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:36.672063 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4bd31f87-047f-45bd-b768-c0e3ff2e2c7f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:36.673372 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-4bd31f87-047f-45bd-b768-c0e3ff2e2c7f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:36.680990 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4bd31f87-047f-45bd-b768-c0e3ff2e2c7f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:36.681474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4bd31f87-047f-45bd-b768-c0e3ff2e2c7f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:36.682251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4bd31f87-047f-45bd-b768-c0e3ff2e2c7f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:36.682683 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-4bd31f87-047f-45bd-b768-c0e3ff2e2c7f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:36.782414 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-4bd31f87-047f-45bd-b768-c0e3ff2e2c7f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:36.793589 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4bd31f87-047f-45bd-b768-c0e3ff2e2c7f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1294 microversion: 2.1 time: 0.124935 May 25 16:44:36.795036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1262/2503] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:36 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1294 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:37.682597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:37.683716 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:37.690723 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:37.691195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:37.692425 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:37.692857 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:37.811975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b3422671-6ece-4629-b6e9-e356e93e307b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:37.812436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-b3422671-6ece-4629-b6e9-e356e93e307b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:37.821207 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b3422671-6ece-4629-b6e9-e356e93e307b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:37.822974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b3422671-6ece-4629-b6e9-e356e93e307b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:37.824518 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b3422671-6ece-4629-b6e9-e356e93e307b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:37.825251 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-b3422671-6ece-4629-b6e9-e356e93e307b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:37.940059 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-b3422671-6ece-4629-b6e9-e356e93e307b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:37.951224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b3422671-6ece-4629-b6e9-e356e93e307b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1300 microversion: 2.1 time: 0.143449 May 25 16:44:37.952613 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1263/2504] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:37 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1300 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:44:37.966444 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:37.975436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:37.977295 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:37.978648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9c22b1aa-6919-4e4e-a99c-1fda4554d9ab tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1407 microversion: 2.20 time: 0.300699 May 25 16:44:37.979835 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1242/2505] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:37 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1407 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:38.970724 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-52bff3ea-a7f6-48a9-b52a-a8fadaea485f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:38.974035 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-52bff3ea-a7f6-48a9-b52a-a8fadaea485f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:38.996717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-52bff3ea-a7f6-48a9-b52a-a8fadaea485f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:38.998224 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:38.998833 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:38.999380 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-52bff3ea-a7f6-48a9-b52a-a8fadaea485f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:39.001850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-52bff3ea-a7f6-48a9-b52a-a8fadaea485f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:39.003627 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-52bff3ea-a7f6-48a9-b52a-a8fadaea485f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:39.020355 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:39.020806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:39.021578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:39.022007 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:39.418186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-52bff3ea-a7f6-48a9-b52a-a8fadaea485f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:39.428569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-52bff3ea-a7f6-48a9-b52a-a8fadaea485f tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1368 microversion: 2.1 time: 0.463593 May 25 16:44:39.439515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1264/2506] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:38 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1368 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:39.453849 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:39.463920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:39.465729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:39.467112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b15e1460-9f18-4969-b33a-f6eed39a601c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1407 microversion: 2.20 time: 0.472887 May 25 16:44:39.468270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1243/2507] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:38 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1407 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:39.606723 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-61ffa27c-040c-4030-b246-909341bcd0a8 req-1a3cca92-9c6b-4485-969c-115e188735ee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c630b84f-2196-42fc-a893-72eb8d3f2b34", "tag": "6f71d2d6-3f3f-4287-836c-01e24c1eaecc"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:39.614689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-61ffa27c-040c-4030-b246-909341bcd0a8 req-1a3cca92-9c6b-4485-969c-115e188735ee service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:39.615130 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-61ffa27c-040c-4030-b246-909341bcd0a8 req-1a3cca92-9c6b-4485-969c-115e188735ee service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:39.626039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-61ffa27c-040c-4030-b246-909341bcd0a8 req-1a3cca92-9c6b-4485-969c-115e188735ee service nova] Creating event network-changed:6f71d2d6-3f3f-4287-836c-01e24c1eaecc for instance c630b84f-2196-42fc-a893-72eb8d3f2b34 on hv-cinder-84327 May 25 16:44:39.632920 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-61ffa27c-040c-4030-b246-909341bcd0a8 req-1a3cca92-9c6b-4485-969c-115e188735ee service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.191761 May 25 16:44:39.633591 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1265/2508] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:44:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:44:40.049471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-3836c7b6-f966-463a-ad99-c816d5f0a359 req-ddae581e-3d96-4a64-80c3-f10c60b62e6b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4af3d63d-392d-4abb-bf38-cc4306f5b631", "tag": "62815ccc-47e9-4214-8402-6dd2e9e1a6bf"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:40.060927 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-3836c7b6-f966-463a-ad99-c816d5f0a359 req-ddae581e-3d96-4a64-80c3-f10c60b62e6b service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:40.062061 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-3836c7b6-f966-463a-ad99-c816d5f0a359 req-ddae581e-3d96-4a64-80c3-f10c60b62e6b service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:40.079587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-3836c7b6-f966-463a-ad99-c816d5f0a359 req-ddae581e-3d96-4a64-80c3-f10c60b62e6b service nova] Creating event network-changed:62815ccc-47e9-4214-8402-6dd2e9e1a6bf for instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 on hv-cinder-84327 May 25 16:44:40.085442 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-3836c7b6-f966-463a-ad99-c816d5f0a359 req-ddae581e-3d96-4a64-80c3-f10c60b62e6b service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.117072 May 25 16:44:40.086404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1244/2509] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:44:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:44:40.447603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-efa01e60-357e-479d-90c8-9918a6eeb3ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:40.448661 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-efa01e60-357e-479d-90c8-9918a6eeb3ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:40.460109 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efa01e60-357e-479d-90c8-9918a6eeb3ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:40.461192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efa01e60-357e-479d-90c8-9918a6eeb3ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:40.462556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efa01e60-357e-479d-90c8-9918a6eeb3ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:40.463530 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-efa01e60-357e-479d-90c8-9918a6eeb3ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:40.487243 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:40.488364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:40.495584 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:40.497089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:40.498176 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:40.498812 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:40.966507 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:40.974954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:40.976814 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:40.978250 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3a44cc1c-acda-4fe4-9121-1a97918507ea tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1407 microversion: 2.20 time: 0.496811 May 25 16:44:40.989020 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1245/2510] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:40 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1407 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:40.998911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-efa01e60-357e-479d-90c8-9918a6eeb3ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:41.007853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-efa01e60-357e-479d-90c8-9918a6eeb3ad tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1368 microversion: 2.1 time: 0.565375 May 25 16:44:41.009861 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1266/2511] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:40 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1368 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:41.997008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:41.997581 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:42.004203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:42.004909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:42.005690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:42.006127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:42.025666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6d0472c1-47b0-45d7-b57e-5adb35bd04b8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:42.027025 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6d0472c1-47b0-45d7-b57e-5adb35bd04b8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:42.034292 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d0472c1-47b0-45d7-b57e-5adb35bd04b8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:42.035333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d0472c1-47b0-45d7-b57e-5adb35bd04b8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:42.036310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d0472c1-47b0-45d7-b57e-5adb35bd04b8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:42.036868 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6d0472c1-47b0-45d7-b57e-5adb35bd04b8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:42.394182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:42.402746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:42.404279 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:42.405400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f7dad338-d2af-493b-b206-9c932ec7794f tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1407 microversion: 2.20 time: 0.412442 May 25 16:44:42.416112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1246/2512] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:41 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1407 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:42.425310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-6d0472c1-47b0-45d7-b57e-5adb35bd04b8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:42.432759 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6d0472c1-47b0-45d7-b57e-5adb35bd04b8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1368 microversion: 2.1 time: 0.411614 May 25 16:44:42.433340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1267/2513] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:42 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1368 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:44:43.424241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:43.424910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:43.432462 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:43.432923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:43.433741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:43.434210 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:43.449973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-0c329c98-61ab-48ce-8823-980aeb00a3f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:43.450621 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-0c329c98-61ab-48ce-8823-980aeb00a3f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:43.463883 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c329c98-61ab-48ce-8823-980aeb00a3f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:43.464323 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c329c98-61ab-48ce-8823-980aeb00a3f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:43.465326 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c329c98-61ab-48ce-8823-980aeb00a3f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:43.465757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-0c329c98-61ab-48ce-8823-980aeb00a3f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:43.764098 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:43.768685 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-0c329c98-61ab-48ce-8823-980aeb00a3f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:43.773554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:43.775402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:43.776704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b1a1b634-3379-4990-a2c7-f31dc608e90d tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1407 microversion: 2.20 time: 0.356565 May 25 16:44:43.778483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-0c329c98-61ab-48ce-8823-980aeb00a3f8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1368 microversion: 2.1 time: 0.332583 May 25 16:44:43.788264 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1247/2515] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:43 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1407 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:43.788448 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1268/2515] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:43 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1368 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:44.004996 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-9ce9b0d8-dbf3-47be-93a4-08b79e38c176 req-62c937b9-1873-4b4d-9150-ccb553e9aff2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c630b84f-2196-42fc-a893-72eb8d3f2b34", "name": "network-vif-plugged", "status": "completed", "tag": "6f71d2d6-3f3f-4287-836c-01e24c1eaecc"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:44.014881 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-9ce9b0d8-dbf3-47be-93a4-08b79e38c176 req-62c937b9-1873-4b4d-9150-ccb553e9aff2 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:44.015682 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-9ce9b0d8-dbf3-47be-93a4-08b79e38c176 req-62c937b9-1873-4b4d-9150-ccb553e9aff2 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:44.028875 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-9ce9b0d8-dbf3-47be-93a4-08b79e38c176 req-62c937b9-1873-4b4d-9150-ccb553e9aff2 service nova] Creating event network-vif-plugged:6f71d2d6-3f3f-4287-836c-01e24c1eaecc for instance c630b84f-2196-42fc-a893-72eb8d3f2b34 on hv-cinder-84327 May 25 16:44:44.036038 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-9ce9b0d8-dbf3-47be-93a4-08b79e38c176 req-62c937b9-1873-4b4d-9150-ccb553e9aff2 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.245720 May 25 16:44:44.037335 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1269/2516] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:44:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:44:44.795128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-36026782-7f30-453b-9809-0ef154fb80f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:44.796411 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-36026782-7f30-453b-9809-0ef154fb80f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:44.798206 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:44.799629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:44.804457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-36026782-7f30-453b-9809-0ef154fb80f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:44.805194 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-36026782-7f30-453b-9809-0ef154fb80f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:44.806125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-36026782-7f30-453b-9809-0ef154fb80f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:44.806741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-36026782-7f30-453b-9809-0ef154fb80f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:44.809066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:44.811184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:44.812255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:44.813887 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:45.118558 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-36026782-7f30-453b-9809-0ef154fb80f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:45.128475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:45.132994 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-36026782-7f30-453b-9809-0ef154fb80f5 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1368 microversion: 2.1 time: 0.341736 May 25 16:44:45.134789 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1248/2517] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:44 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1368 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:45.141249 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:45.143046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:45.144400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ec5706b1-3523-4408-8ecd-fed617f84688 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1407 microversion: 2.20 time: 0.352234 May 25 16:44:45.145384 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1270/2518] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:44 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1407 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:46.163420 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:46.164853 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-1e9d0f9e-2f2e-4d8b-a4b8-1dd0581e933b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:46.165992 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-1e9d0f9e-2f2e-4d8b-a4b8-1dd0581e933b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:46.166792 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:46.169556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:46.173236 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=12 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:46.180855 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:46.185844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=12 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:46.191919 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e9d0f9e-2f2e-4d8b-a4b8-1dd0581e933b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:46.192449 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e9d0f9e-2f2e-4d8b-a4b8-1dd0581e933b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:46.193275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e9d0f9e-2f2e-4d8b-a4b8-1dd0581e933b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:46.195906 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:46.196994 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:46.198717 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:46.200739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-1e9d0f9e-2f2e-4d8b-a4b8-1dd0581e933b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:46.203145 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:47.074899 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-1e9d0f9e-2f2e-4d8b-a4b8-1dd0581e933b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:47.106602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-1e9d0f9e-2f2e-4d8b-a4b8-1dd0581e933b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1368 microversion: 2.1 time: 0.956676 May 25 16:44:47.119753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1249/2519] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:46 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1368 bytes in 970 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:47.151226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:47.174216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:47.177708 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:47.178938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-512ec659-8322-41df-80f6-10ac76d374f6 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1599 microversion: 2.20 time: 1.020210 May 25 16:44:47.192228 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1271/2520] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:44:46 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1599 bytes in 1034 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:44:47.198692 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-527ce8ab-14b9-4932-8ac8-45126f170ea2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:47.199535 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-527ce8ab-14b9-4932-8ac8-45126f170ea2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:47.213172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-527ce8ab-14b9-4932-8ac8-45126f170ea2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:47.213858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-527ce8ab-14b9-4932-8ac8-45126f170ea2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:47.216756 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-527ce8ab-14b9-4932-8ac8-45126f170ea2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:47.217323 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-527ce8ab-14b9-4932-8ac8-45126f170ea2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:47.254407 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=100 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:44:47.416426 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-527ce8ab-14b9-4932-8ac8-45126f170ea2 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.222172 May 25 16:44:47.417343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1250/2521] 10.222.0.45 () {60 vars in 1315 bytes} [Wed May 25 16:44:47 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34/os-interface => generated 284 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:44:47.487513 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-f77f39a0-a2eb-428d-ad31-8610b8ca5bc4 req-28c83a67-4adf-42da-b272-09678a327608 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4af3d63d-392d-4abb-bf38-cc4306f5b631", "name": "network-vif-plugged", "status": "completed", "tag": "62815ccc-47e9-4214-8402-6dd2e9e1a6bf"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:47.503502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-f77f39a0-a2eb-428d-ad31-8610b8ca5bc4 req-28c83a67-4adf-42da-b272-09678a327608 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:47.503938 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-f77f39a0-a2eb-428d-ad31-8610b8ca5bc4 req-28c83a67-4adf-42da-b272-09678a327608 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:47.516727 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-f77f39a0-a2eb-428d-ad31-8610b8ca5bc4 req-28c83a67-4adf-42da-b272-09678a327608 service nova] Creating event network-vif-plugged:62815ccc-47e9-4214-8402-6dd2e9e1a6bf for instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 on hv-cinder-84327 May 25 16:44:47.523099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-f77f39a0-a2eb-428d-ad31-8610b8ca5bc4 req-28c83a67-4adf-42da-b272-09678a327608 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043534 May 25 16:44:47.524356 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1272/2522] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:44:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:44:48.128993 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-98cf4f79-943c-4b35-ad37-312f6ed0c399 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:48.129677 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-98cf4f79-943c-4b35-ad37-312f6ed0c399 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:48.146672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98cf4f79-943c-4b35-ad37-312f6ed0c399 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:48.147587 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98cf4f79-943c-4b35-ad37-312f6ed0c399 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:48.148660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98cf4f79-943c-4b35-ad37-312f6ed0c399 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:48.149534 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-98cf4f79-943c-4b35-ad37-312f6ed0c399 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:48.455974 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-98cf4f79-943c-4b35-ad37-312f6ed0c399 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:48.468916 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-98cf4f79-943c-4b35-ad37-312f6ed0c399 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1368 microversion: 2.1 time: 0.348654 May 25 16:44:48.469843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1251/2523] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:48 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1368 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:48.873901 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-a3f0c043-742b-456b-b0e9-69ca7a80f1c5 req-e5922b97-6a4e-4884-83fc-6eb33526a0bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c630b84f-2196-42fc-a893-72eb8d3f2b34", "tag": "6f71d2d6-3f3f-4287-836c-01e24c1eaecc"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:48.889709 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-a3f0c043-742b-456b-b0e9-69ca7a80f1c5 req-e5922b97-6a4e-4884-83fc-6eb33526a0bf service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:48.898428 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-a3f0c043-742b-456b-b0e9-69ca7a80f1c5 req-e5922b97-6a4e-4884-83fc-6eb33526a0bf service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:48.926511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-a3f0c043-742b-456b-b0e9-69ca7a80f1c5 req-e5922b97-6a4e-4884-83fc-6eb33526a0bf service nova] Creating event network-changed:6f71d2d6-3f3f-4287-836c-01e24c1eaecc for instance c630b84f-2196-42fc-a893-72eb8d3f2b34 on hv-cinder-84327 May 25 16:44:48.940079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-a3f0c043-742b-456b-b0e9-69ca7a80f1c5 req-e5922b97-6a4e-4884-83fc-6eb33526a0bf service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071090 May 25 16:44:48.941375 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1273/2524] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:44:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:44:49.324392 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8ce9c06d-6642-4b1c-ace3-e1ddfff295b6 tempest-TestShelveInstance-1816249359 tempest-TestShelveInstance-1816249359-project] Action: 'create', calling method: .version_select at 0x7f4a3004f9d0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1835458511", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBENb3I31yuR+OIUzv6SKuL9AM8btCXO66X+hJQaqlyzONZSpRUx0D4X8Q1g65UrqAyfhX5dLW2tVdnBEfg5zIQRF0tqORpKp5Zt7wRTq3UwKgWYiiOQ4tsEzZugGF0cxHw=="}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:49.338204 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-8ce9c06d-6642-4b1c-ace3-e1ddfff295b6 tempest-TestShelveInstance-1816249359 tempest-TestShelveInstance-1816249359-project] Getting quotas for project dfd05f6349194d52b2a23b550cae2105. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:44:49.341945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-8ce9c06d-6642-4b1c-ace3-e1ddfff295b6 tempest-TestShelveInstance-1816249359 tempest-TestShelveInstance-1816249359-project] Getting quotas for user d7d9118b197b44aebe1d8d9abe52f58f and project dfd05f6349194d52b2a23b550cae2105. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:44:49.363858 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8ce9c06d-6642-4b1c-ace3-e1ddfff295b6 tempest-TestShelveInstance-1816249359 tempest-TestShelveInstance-1816249359-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.146405 May 25 16:44:49.370009 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1252/2525] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:44:49 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:44:49.486872 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ce6a52ae-7304-4b45-9eef-7aa733802bec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:49.487853 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ce6a52ae-7304-4b45-9eef-7aa733802bec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:49.496377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce6a52ae-7304-4b45-9eef-7aa733802bec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:49.497192 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce6a52ae-7304-4b45-9eef-7aa733802bec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:49.498219 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce6a52ae-7304-4b45-9eef-7aa733802bec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:49.499462 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ce6a52ae-7304-4b45-9eef-7aa733802bec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:49.717086 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ce6a52ae-7304-4b45-9eef-7aa733802bec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:49.745983 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ce6a52ae-7304-4b45-9eef-7aa733802bec tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1368 microversion: 2.1 time: 0.264238 May 25 16:44:49.747174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1274/2526] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:49 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1368 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:50.764260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dbb0b6fa-0094-49bc-bf51-5a7efc5716c2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:50.764834 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dbb0b6fa-0094-49bc-bf51-5a7efc5716c2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:50.772268 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dbb0b6fa-0094-49bc-bf51-5a7efc5716c2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:50.772709 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dbb0b6fa-0094-49bc-bf51-5a7efc5716c2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:50.773485 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dbb0b6fa-0094-49bc-bf51-5a7efc5716c2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:50.773901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dbb0b6fa-0094-49bc-bf51-5a7efc5716c2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:50.915070 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dbb0b6fa-0094-49bc-bf51-5a7efc5716c2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:44:50.923688 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dbb0b6fa-0094-49bc-bf51-5a7efc5716c2 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1558 microversion: 2.1 time: 0.163848 May 25 16:44:50.927193 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1253/2527] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:44:50 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:44:50.940036 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-2dda73f9-259e-44bc-bdb8-e76b45856dac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:44:50.940570 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-2dda73f9-259e-44bc-bdb8-e76b45856dac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:44:50.947611 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2dda73f9-259e-44bc-bdb8-e76b45856dac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:50.948353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2dda73f9-259e-44bc-bdb8-e76b45856dac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:50.949117 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2dda73f9-259e-44bc-bdb8-e76b45856dac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:50.949543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-2dda73f9-259e-44bc-bdb8-e76b45856dac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:51.036474 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-2dda73f9-259e-44bc-bdb8-e76b45856dac tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.100271 May 25 16:44:51.037087 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1275/2528] 10.222.0.45 () {58 vars in 1274 bytes} [Wed May 25 16:44:50 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631/os-interface => generated 284 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:44:52.080890 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-473cb79e-707f-4145-b494-a472ef044a2b req-000e7462-5756-4152-bbcd-b48397a40372 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4af3d63d-392d-4abb-bf38-cc4306f5b631", "tag": "62815ccc-47e9-4214-8402-6dd2e9e1a6bf"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:44:52.090388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-473cb79e-707f-4145-b494-a472ef044a2b req-000e7462-5756-4152-bbcd-b48397a40372 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:44:52.090843 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-473cb79e-707f-4145-b494-a472ef044a2b req-000e7462-5756-4152-bbcd-b48397a40372 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:44:52.102458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-473cb79e-707f-4145-b494-a472ef044a2b req-000e7462-5756-4152-bbcd-b48397a40372 service nova] Creating event network-changed:62815ccc-47e9-4214-8402-6dd2e9e1a6bf for instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 on hv-cinder-84327 May 25 16:44:52.107910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-473cb79e-707f-4145-b494-a472ef044a2b req-000e7462-5756-4152-bbcd-b48397a40372 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030819 May 25 16:44:52.108497 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1254/2529] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:44:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:45:23.327017 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:INSERT=1,nova_api:SELECT=32 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:45:23.328119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=73 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:45:23.335226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-3b503147-a62f-4ada-aa4a-e78c07021913 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:45:23.336072 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-3b503147-a62f-4ada-aa4a-e78c07021913 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:45:23.344173 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b503147-a62f-4ada-aa4a-e78c07021913 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:45:23.344615 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b503147-a62f-4ada-aa4a-e78c07021913 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:45:23.345359 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b503147-a62f-4ada-aa4a-e78c07021913 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:45:23.345784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-3b503147-a62f-4ada-aa4a-e78c07021913 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:45:23.369624 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-3b503147-a62f-4ada-aa4a-e78c07021913 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34/ips" status: 200 len: 152 microversion: 2.20 time: 0.046098 May 25 16:45:23.370184 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1276/2530] 10.222.0.45 () {60 vars in 1288 bytes} [Wed May 25 16:45:23 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34/ips => generated 152 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 25 16:45:25.989956 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d20a588d-b88f-408d-9b30-3fd99129e609 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:45:25.990661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-d20a588d-b88f-408d-9b30-3fd99129e609 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:45:25.993092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=4 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:45:25.993832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=4 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:45:25.994510 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=15 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:45:26.001999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d20a588d-b88f-408d-9b30-3fd99129e609 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:45:26.002490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d20a588d-b88f-408d-9b30-3fd99129e609 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:45:26.003358 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d20a588d-b88f-408d-9b30-3fd99129e609 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:45:26.003821 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-d20a588d-b88f-408d-9b30-3fd99129e609 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:45:26.028471 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d20a588d-b88f-408d-9b30-3fd99129e609 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631/ips" status: 200 len: 149 microversion: 2.1 time: 0.043231 May 25 16:45:26.029024 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1255/2531] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:45:25 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631/ips => generated 149 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:45:50.815957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ebafc123-e15d-4481-8171-e47b42ff31d2 tempest-TestShelveInstance-1816249359 tempest-TestShelveInstance-1816249359-project] Calling method '.version_select at 0x7f4a351b0040>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:45:50.823537 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:45:50.824407 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=4 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:45:50.840992 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ebafc123-e15d-4481-8171-e47b42ff31d2 tempest-TestShelveInstance-1816249359 tempest-TestShelveInstance-1816249359-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1835458511" status: 202 len: 0 microversion: 2.1 time: 0.030757 May 25 16:45:50.841957 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1277/2532] 10.222.0.45 () {58 vars in 1253 bytes} [Wed May 25 16:45:50 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1835458511 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:46:23.457725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ad9b1f06-dded-4bda-b503-1e72fab688b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:23.457725 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ad9b1f06-dded-4bda-b503-1e72fab688b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:23.460653 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:46:23.461348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=4 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:46:23.468739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad9b1f06-dded-4bda-b503-1e72fab688b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:23.469186 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad9b1f06-dded-4bda-b503-1e72fab688b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:23.469943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad9b1f06-dded-4bda-b503-1e72fab688b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:23.470375 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ad9b1f06-dded-4bda-b503-1e72fab688b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:23.496256 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ad9b1f06-dded-4bda-b503-1e72fab688b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:46:23.580439 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ad9b1f06-dded-4bda-b503-1e72fab688b1 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "DELETE /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 204 len: 0 microversion: 2.20 time: 0.130959 May 25 16:46:23.581124 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1256/2533] 10.222.0.45 () {60 vars in 1279 bytes} [Wed May 25 16:46:23 2022] DELETE /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 25 16:46:23.587065 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=3,nova_api:DELETE=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:46:23.591440 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:23.592050 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:23.601915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:23.602371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:23.603131 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:23.603603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:23.741421 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:23.749705 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:23.751186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:23.752391 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c6f0459e-fd9d-4913-a1dc-9d1e6eea9395 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1724 microversion: 2.20 time: 0.166733 May 25 16:46:23.752946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1278/2534] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:46:23 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1724 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:46:24.774745 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:24.775663 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:24.783746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:24.784196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:24.784952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:24.785382 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:24.968660 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:24.978990 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:24.980500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:24.981689 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f69ce57e-fef2-441c-8da2-bec504eed62c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1724 microversion: 2.20 time: 0.213916 May 25 16:46:24.982236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1257/2535] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:46:24 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1724 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:46:25.795245 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-2eb30f05-6a70-45ef-b547-3b539de171f4 req-48de6077-f7fe-40cb-8ea4-d8c42d8a5003 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c630b84f-2196-42fc-a893-72eb8d3f2b34", "name": "network-vif-deleted", "tag": "6f71d2d6-3f3f-4287-836c-01e24c1eaecc"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:46:25.804410 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-2eb30f05-6a70-45ef-b547-3b539de171f4 req-48de6077-f7fe-40cb-8ea4-d8c42d8a5003 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:25.804890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-2eb30f05-6a70-45ef-b547-3b539de171f4 req-48de6077-f7fe-40cb-8ea4-d8c42d8a5003 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:25.816914 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-2eb30f05-6a70-45ef-b547-3b539de171f4 req-48de6077-f7fe-40cb-8ea4-d8c42d8a5003 service nova] Creating event network-vif-deleted:6f71d2d6-3f3f-4287-836c-01e24c1eaecc for instance c630b84f-2196-42fc-a893-72eb8d3f2b34 on hv-cinder-84327 May 25 16:46:25.822539 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-2eb30f05-6a70-45ef-b547-3b539de171f4 req-48de6077-f7fe-40cb-8ea4-d8c42d8a5003 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031194 May 25 16:46:25.826432 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1279/2536] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:46:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:46:26.007205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:26.008493 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:26.017055 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:26.017500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:26.019400 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:26.020132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:26.069421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:26.070168 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:26.070920 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lazy-loading 'fault' on Instance uuid c630b84f-2196-42fc-a893-72eb8d3f2b34 {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:46:26.108302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d700851a-6f92-42db-968c-af0cfbe22dbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:26.108839 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d700851a-6f92-42db-968c-af0cfbe22dbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:26.115669 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d700851a-6f92-42db-968c-af0cfbe22dbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:26.116204 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d700851a-6f92-42db-968c-af0cfbe22dbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:26.117048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d700851a-6f92-42db-968c-af0cfbe22dbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:26.117554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d700851a-6f92-42db-968c-af0cfbe22dbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:26.118151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:26.127728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:26.129364 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '06ba885a0b164134898c42a332feba76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c3003077416c4ce997e3aa7e47e8e3b7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:26.131111 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ece8b832-e89f-411f-883f-a4af048b355c tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 200 len: 1371 microversion: 2.20 time: 0.127758 May 25 16:46:26.143236 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1258/2537] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:46:25 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 1371 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 25 16:46:26.156353 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d700851a-6f92-42db-968c-af0cfbe22dbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Going to try to terminate instance {{(pid=117795) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:46:26.237985 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d700851a-6f92-42db-968c-af0cfbe22dbe tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 204 len: 0 microversion: 2.1 time: 0.133931 May 25 16:46:26.238693 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1280/2538] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:46:26 2022] DELETE /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:46:26.247597 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-8a600175-0fe7-4e94-8240-384cb1782774 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:26.248228 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-8a600175-0fe7-4e94-8240-384cb1782774 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:26.256546 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a600175-0fe7-4e94-8240-384cb1782774 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:26.257000 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a600175-0fe7-4e94-8240-384cb1782774 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:26.258388 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a600175-0fe7-4e94-8240-384cb1782774 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:26.258826 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-8a600175-0fe7-4e94-8240-384cb1782774 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:26.405602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-8a600175-0fe7-4e94-8240-384cb1782774 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:26.413871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-8a600175-0fe7-4e94-8240-384cb1782774 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1682 microversion: 2.1 time: 0.170374 May 25 16:46:26.414526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1259/2539] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:26 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1682 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:27.154390 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:27.155095 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:27.161877 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:27.162327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:27.163090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:27.163880 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:27.178168 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] HTTP exception thrown: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Request method failure captured: May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 HTTP/1.1 May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaS4R2Q0uyyQiC_zqcGI-JA7bCVvbPwrz_330b_JgHbpSrVu9wM-WpOvctaejhwGxpAodwx3WJWsuQ0GFkUbHXQCzUkq6Fe6YZKZ15x3DYZvoWLLf8QyMPtbUZHDXgGJ7Z2fImgS--Ix-P6gbsMoij8yzFICYuR4jTIofo8HNi4SqA May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Openstack-Nova-Api-Version: 2.20 May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:46:27.179869 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 06ba885a0b164134898c42a332feba76 May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:27.181845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'c630b84f-2196-42fc-a893-72eb8d3f2b34'} May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:46:27.182969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:27.184475 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:27.185775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Returning 404 to user: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:46:27.185775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-a23f7adc-d332-4917-9b40-afae3b221063 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 404 len: 111 microversion: 2.20 time: 0.032701 May 25 16:46:27.185775 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1281/2540] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:46:27 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 25 16:46:27.193129 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-067afc26-5c41-4692-8df6-bfce4b075b73 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '.version_select at 0x7f4a300fa5e0>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:27.218841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-067afc26-5c41-4692-8df6-bfce4b075b73 tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1949390145" status: 204 len: 0 microversion: 2.20 time: 0.031980 May 25 16:46:27.219512 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1260/2541] 10.222.0.45 () {60 vars in 1261 bytes} [Wed May 25 16:46:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1949390145 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) May 25 16:46:27.431755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1d61bc34-665f-40cb-a97e-d9750626efa0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:27.433257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1d61bc34-665f-40cb-a97e-d9750626efa0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:27.439404 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d61bc34-665f-40cb-a97e-d9750626efa0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:27.439848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d61bc34-665f-40cb-a97e-d9750626efa0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:27.440596 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d61bc34-665f-40cb-a97e-d9750626efa0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:27.441023 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1d61bc34-665f-40cb-a97e-d9750626efa0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:27.708932 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-1d61bc34-665f-40cb-a97e-d9750626efa0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:27.717043 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1d61bc34-665f-40cb-a97e-d9750626efa0 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1682 microversion: 2.1 time: 0.289248 May 25 16:46:27.717706 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1282/2542] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:27 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1682 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:28.261856 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:28.262436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:28.269508 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:28.269952 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:28.270697 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:28.271122 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:28.284205 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] HTTP exception thrown: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Request method failure captured: May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: DELETE /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 HTTP/1.1 May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaS4R2Q0uyyQiC_zqcGI-JA7bCVvbPwrz_330b_JgHbpSrVu9wM-WpOvctaejhwGxpAodwx3WJWsuQ0GFkUbHXQCzUkq6Fe6YZKZ15x3DYZvoWLLf8QyMPtbUZHDXgGJ7Z2fImgS--Ix-P6gbsMoij8yzFICYuR4jTIofo8HNi4SqA May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Openstack-Nova-Api-Version: 2.20 May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:46:28.286108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 06ba885a0b164134898c42a332feba76 May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.287884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'c630b84f-2196-42fc-a893-72eb8d3f2b34'} May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:46:28.289081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.290491 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:28.292133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Returning 404 to user: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:46:28.292133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b12ef4ac-06f9-45fc-a431-ceec8a8a26fe tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "DELETE /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 404 len: 111 microversion: 2.20 time: 0.030856 May 25 16:46:28.292133 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1261/2543] 10.222.0.45 () {60 vars in 1279 bytes} [Wed May 25 16:46:28 2022] DELETE /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 25 16:46:28.296569 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:28.297166 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: c630b84f-2196-42fc-a893-72eb8d3f2b34] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:28.303663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:28.304105 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:28.304852 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:28.305287 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:28.318641 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] HTTP exception thrown: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Request method failure captured: May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 HTTP/1.1 May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaS4R2Q0uyyQiC_zqcGI-JA7bCVvbPwrz_330b_JgHbpSrVu9wM-WpOvctaejhwGxpAodwx3WJWsuQ0GFkUbHXQCzUkq6Fe6YZKZ15x3DYZvoWLLf8QyMPtbUZHDXgGJ7Z2fImgS--Ix-P6gbsMoij8yzFICYuR4jTIofo8HNi4SqA May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Openstack-Nova-Api-Version: 2.20 May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:46:28.320191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 06ba885a0b164134898c42a332feba76 May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.322290 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'c630b84f-2196-42fc-a893-72eb8d3f2b34'} May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:46:28.323203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. May 25 16:46:28.324607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:46:28.326367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Returning 404 to user: Instance c630b84f-2196-42fc-a893-72eb8d3f2b34 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:46:28.326367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c359a3ab-e304-4ee1-9c3b-d55068c106ad tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34" status: 404 len: 111 microversion: 2.20 time: 0.028747 May 25 16:46:28.326367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1283/2544] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:46:28 2022] GET /compute/v2.1/servers/c630b84f-2196-42fc-a893-72eb8d3f2b34 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 25 16:46:28.694116 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-2f452494-3f86-411f-9679-bcd3bd590254 req-ea87da8b-338c-4895-9938-bb35f667ea64 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4af3d63d-392d-4abb-bf38-cc4306f5b631", "name": "network-vif-deleted", "tag": "62815ccc-47e9-4214-8402-6dd2e9e1a6bf"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:46:28.702593 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-2f452494-3f86-411f-9679-bcd3bd590254 req-ea87da8b-338c-4895-9938-bb35f667ea64 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:28.703049 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-2f452494-3f86-411f-9679-bcd3bd590254 req-ea87da8b-338c-4895-9938-bb35f667ea64 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:28.718643 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-2f452494-3f86-411f-9679-bcd3bd590254 req-ea87da8b-338c-4895-9938-bb35f667ea64 service nova] Creating event network-vif-deleted:62815ccc-47e9-4214-8402-6dd2e9e1a6bf for instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 on hv-cinder-84327 May 25 16:46:28.723682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-2f452494-3f86-411f-9679-bcd3bd590254 req-ea87da8b-338c-4895-9938-bb35f667ea64 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030391 May 25 16:46:28.723682 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1262/2545] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:46:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:46:28.741637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-15353488-115a-4219-8782-04dd748b1cfa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:28.742302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-15353488-115a-4219-8782-04dd748b1cfa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:28.748981 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15353488-115a-4219-8782-04dd748b1cfa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:28.749277 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15353488-115a-4219-8782-04dd748b1cfa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:28.750272 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15353488-115a-4219-8782-04dd748b1cfa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:28.750725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-15353488-115a-4219-8782-04dd748b1cfa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:28.844226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-15353488-115a-4219-8782-04dd748b1cfa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:28.859230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-15353488-115a-4219-8782-04dd748b1cfa tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 200 len: 1327 microversion: 2.1 time: 0.125991 May 25 16:46:28.859737 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1284/2546] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:28 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 1327 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:29.876686 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:29.877274 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:29.884495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:29.884944 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:29.885819 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:29.886246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:29.901317 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 HTTP/1.1 May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:46:29.903011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:46:29.904672 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '4af3d63d-392d-4abb-bf38-cc4306f5b631'} May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:46:29.905943 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:46:29.907589 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:46:29.909172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:46:29.909172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c461a9fa-1d87-435b-98be-882e280d067c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 404 len: 111 microversion: 2.1 time: 0.032057 May 25 16:46:29.909172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1263/2547] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:29 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:46:29.915093 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-45c06909-2b2f-417a-a415-450be2d64892 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '.version_select at 0x7f4a302443a0>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:29.936280 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-45c06909-2b2f-417a-a415-450be2d64892 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1636892488" status: 202 len: 0 microversion: 2.1 time: 0.024930 May 25 16:46:29.937269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1285/2548] 10.222.0.45 () {58 vars in 1220 bytes} [Wed May 25 16:46:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1636892488 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:46:30.604967 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-887aa35d-597e-462b-9da4-be6b71d3698d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: .version_select at 0x7f4a237e25e0>, body: {"keypair": {"name": "tempest-keypair-1925197884", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAJuA1zrI0IQIEdHDstYpy3MgdJdoIc8yqa6hd9PZGWn5+wP6PZW5/4yQjuXneCFARSjE7kRc+qAolbUwpAK4E+23ffCyqBX7cenzFE8gB/JBGUkURNGKqEkqDVojYB3VA=="}} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:46:30.620728 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-887aa35d-597e-462b-9da4-be6b71d3698d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:46:30.624759 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.quota [None req-887aa35d-597e-462b-9da4-be6b71d3698d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for user a0c9f3f10fd142439521e4b36cf33157 and project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'key_pairs'} {{(pid=117794) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:46:30.649492 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-887aa35d-597e-462b-9da4-be6b71d3698d tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.049080 May 25 16:46:30.650089 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1264/2549] 10.222.0.45 () {60 vars in 1158 bytes} [Wed May 25 16:46:30 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:46:32.003698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-500266078", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "aaf08a79-3442-476f-b613-a02c12c36d8d"}], "security_groups": [{"name": "tempest-securitygroup--2102394380"}], "key_name": "tempest-keypair-1925197884", "user_data": "}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:46:32.111496 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] validate_networks() for [('aaf08a79-3442-476f-b613-a02c12c36d8d', None, None, None, None, None)] {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:46:32.487915 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:46:32.488344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:46:32.488816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:46:32.489226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:46:32.489638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:46:32.499129 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:32.499574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:32.500638 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:32.500971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:32.519476 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:46:32.523637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Getting quotas for user a0c9f3f10fd142439521e4b36cf33157 and project ec1d676becac4112a7c0c319e54b2cd2. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:46:32.535214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:46:32.537546 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:46:32.589896 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d4832e81-de4b-47fc-a2f3-19dd2ed76fe6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.590477 May 25 16:46:32.590720 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1286/2550] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:46:31 2022] POST /compute/v2.1/servers => generated 402 bytes in 592 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:46:32.601195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-01e88157-e79b-4034-b5c9-7abd654e5c5b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:32.601763 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-01e88157-e79b-4034-b5c9-7abd654e5c5b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:32.656941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-01e88157-e79b-4034-b5c9-7abd654e5c5b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:32.664429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-01e88157-e79b-4034-b5c9-7abd654e5c5b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1241 microversion: 2.1 time: 0.067610 May 25 16:46:32.665013 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1265/2551] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:32 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1241 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:33.681919 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-59320a64-fc97-4550-9dce-b708821fac67 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:33.682601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-59320a64-fc97-4550-9dce-b708821fac67 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:33.690785 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59320a64-fc97-4550-9dce-b708821fac67 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:33.691545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59320a64-fc97-4550-9dce-b708821fac67 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:33.692205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59320a64-fc97-4550-9dce-b708821fac67 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:33.692645 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59320a64-fc97-4550-9dce-b708821fac67 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:33.759636 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-59320a64-fc97-4550-9dce-b708821fac67 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:33.767443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-59320a64-fc97-4550-9dce-b708821fac67 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1299 microversion: 2.1 time: 0.089698 May 25 16:46:33.768018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1287/2552] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:33 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1299 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:34.784950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-246492a9-816c-4749-aff5-0ba94d80dd93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:34.785583 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-246492a9-816c-4749-aff5-0ba94d80dd93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:34.793628 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-246492a9-816c-4749-aff5-0ba94d80dd93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:34.794244 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-246492a9-816c-4749-aff5-0ba94d80dd93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:34.795361 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-246492a9-816c-4749-aff5-0ba94d80dd93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:34.795883 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-246492a9-816c-4749-aff5-0ba94d80dd93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:35.011779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-246492a9-816c-4749-aff5-0ba94d80dd93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:35.011779 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-246492a9-816c-4749-aff5-0ba94d80dd93 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1367 microversion: 2.1 time: 0.229783 May 25 16:46:35.020100 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1266/2553] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:34 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1367 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:35.345898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-e88fbde5-a0d0-4bf8-8078-20ecd7f87f69 req-9f86ab5a-20ed-494c-a0f5-9fa0a3469ca4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "54076037-ad26-40b3-9791-13f7b1aa56fd", "tag": "e978b244-ff54-4356-8f47-d64ca37af707"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:46:35.363465 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-e88fbde5-a0d0-4bf8-8078-20ecd7f87f69 req-9f86ab5a-20ed-494c-a0f5-9fa0a3469ca4 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:35.364436 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-e88fbde5-a0d0-4bf8-8078-20ecd7f87f69 req-9f86ab5a-20ed-494c-a0f5-9fa0a3469ca4 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:35.388561 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-e88fbde5-a0d0-4bf8-8078-20ecd7f87f69 req-9f86ab5a-20ed-494c-a0f5-9fa0a3469ca4 service nova] Creating event network-changed:e978b244-ff54-4356-8f47-d64ca37af707 for instance 54076037-ad26-40b3-9791-13f7b1aa56fd on hv-cinder-84327 May 25 16:46:35.396202 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-e88fbde5-a0d0-4bf8-8078-20ecd7f87f69 req-9f86ab5a-20ed-494c-a0f5-9fa0a3469ca4 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054167 May 25 16:46:35.397328 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1288/2554] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:46:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:46:36.027950 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-cb0e4a7a-0ea7-4488-8aa3-f0df512392b4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:36.028590 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-cb0e4a7a-0ea7-4488-8aa3-f0df512392b4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:36.035722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb0e4a7a-0ea7-4488-8aa3-f0df512392b4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:36.036169 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb0e4a7a-0ea7-4488-8aa3-f0df512392b4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:36.036914 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb0e4a7a-0ea7-4488-8aa3-f0df512392b4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:36.037523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-cb0e4a7a-0ea7-4488-8aa3-f0df512392b4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:36.195840 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-cb0e4a7a-0ea7-4488-8aa3-f0df512392b4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:36.203482 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-cb0e4a7a-0ea7-4488-8aa3-f0df512392b4 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1367 microversion: 2.1 time: 0.179723 May 25 16:46:36.204252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1267/2555] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:36 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1367 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:37.222048 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ab554717-886a-4d54-955c-0e0999f1ae50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:37.222786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ab554717-886a-4d54-955c-0e0999f1ae50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:37.229763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ab554717-886a-4d54-955c-0e0999f1ae50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:37.230216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ab554717-886a-4d54-955c-0e0999f1ae50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:37.230982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ab554717-886a-4d54-955c-0e0999f1ae50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:37.231448 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ab554717-886a-4d54-955c-0e0999f1ae50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:37.366577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ab554717-886a-4d54-955c-0e0999f1ae50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:37.378986 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ab554717-886a-4d54-955c-0e0999f1ae50 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1367 microversion: 2.1 time: 0.161518 May 25 16:46:37.379630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1289/2556] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:37 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1367 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:38.401234 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b15488ae-2ce3-43a1-948b-31c32b3a0b15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:38.402135 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b15488ae-2ce3-43a1-948b-31c32b3a0b15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:38.409540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b15488ae-2ce3-43a1-948b-31c32b3a0b15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:38.409977 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b15488ae-2ce3-43a1-948b-31c32b3a0b15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:38.410739 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b15488ae-2ce3-43a1-948b-31c32b3a0b15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:38.411188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b15488ae-2ce3-43a1-948b-31c32b3a0b15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:38.555181 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b15488ae-2ce3-43a1-948b-31c32b3a0b15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:38.562760 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b15488ae-2ce3-43a1-948b-31c32b3a0b15 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1367 microversion: 2.1 time: 0.167132 May 25 16:46:38.563448 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1268/2557] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:38 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1367 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:39.587543 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d9d44de0-8d83-4559-8d9b-30102c376e48 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:39.588515 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d9d44de0-8d83-4559-8d9b-30102c376e48 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:39.598185 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d9d44de0-8d83-4559-8d9b-30102c376e48 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:39.598897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d9d44de0-8d83-4559-8d9b-30102c376e48 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:39.600099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d9d44de0-8d83-4559-8d9b-30102c376e48 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:39.600770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d9d44de0-8d83-4559-8d9b-30102c376e48 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:39.740491 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-d9d44de0-8d83-4559-8d9b-30102c376e48 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:39.748614 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d9d44de0-8d83-4559-8d9b-30102c376e48 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1367 microversion: 2.1 time: 0.167854 May 25 16:46:39.749183 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1290/2558] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:39 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1367 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:39.881245 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-b4b1c3a3-c70c-4482-9f06-9fe3067fb851 req-4803fe25-0716-4d03-b927-da117c2c2017 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54076037-ad26-40b3-9791-13f7b1aa56fd", "name": "network-vif-plugged", "status": "completed", "tag": "e978b244-ff54-4356-8f47-d64ca37af707"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:46:39.890913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-b4b1c3a3-c70c-4482-9f06-9fe3067fb851 req-4803fe25-0716-4d03-b927-da117c2c2017 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:39.891486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-b4b1c3a3-c70c-4482-9f06-9fe3067fb851 req-4803fe25-0716-4d03-b927-da117c2c2017 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:39.904704 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-b4b1c3a3-c70c-4482-9f06-9fe3067fb851 req-4803fe25-0716-4d03-b927-da117c2c2017 service nova] Creating event network-vif-plugged:e978b244-ff54-4356-8f47-d64ca37af707 for instance 54076037-ad26-40b3-9791-13f7b1aa56fd on hv-cinder-84327 May 25 16:46:39.911037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-b4b1c3a3-c70c-4482-9f06-9fe3067fb851 req-4803fe25-0716-4d03-b927-da117c2c2017 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033684 May 25 16:46:39.912844 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1269/2559] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:46:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:46:40.770212 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-82b4f0b5-1232-4779-83b5-4ba61e55f4e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:40.771111 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-82b4f0b5-1232-4779-83b5-4ba61e55f4e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:40.779898 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-82b4f0b5-1232-4779-83b5-4ba61e55f4e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:40.780356 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-82b4f0b5-1232-4779-83b5-4ba61e55f4e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:40.781123 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-82b4f0b5-1232-4779-83b5-4ba61e55f4e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:40.781560 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-82b4f0b5-1232-4779-83b5-4ba61e55f4e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:40.918012 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-82b4f0b5-1232-4779-83b5-4ba61e55f4e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:40.929698 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-82b4f0b5-1232-4779-83b5-4ba61e55f4e3 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1367 microversion: 2.1 time: 0.165935 May 25 16:46:40.930485 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1291/2560] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:40 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1367 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:41.948774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-78f10a23-10a6-4fdb-9aa1-96b28f723235 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:41.949429 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-78f10a23-10a6-4fdb-9aa1-96b28f723235 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:41.956627 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78f10a23-10a6-4fdb-9aa1-96b28f723235 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:41.957075 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78f10a23-10a6-4fdb-9aa1-96b28f723235 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:41.957830 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78f10a23-10a6-4fdb-9aa1-96b28f723235 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:41.958260 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-78f10a23-10a6-4fdb-9aa1-96b28f723235 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:42.094123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-78f10a23-10a6-4fdb-9aa1-96b28f723235 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:46:42.101901 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-78f10a23-10a6-4fdb-9aa1-96b28f723235 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1558 microversion: 2.1 time: 0.157416 May 25 16:46:42.102436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1270/2561] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:46:41 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:46:42.118232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9a0898b3-ea7e-4c44-9997-28b453a51084 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:46:42.118739 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9a0898b3-ea7e-4c44-9997-28b453a51084 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:46:42.125631 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a0898b3-ea7e-4c44-9997-28b453a51084 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:42.126079 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a0898b3-ea7e-4c44-9997-28b453a51084 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:42.126836 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a0898b3-ea7e-4c44-9997-28b453a51084 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:42.127413 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9a0898b3-ea7e-4c44-9997-28b453a51084 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:42.207663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9a0898b3-ea7e-4c44-9997-28b453a51084 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.093164 May 25 16:46:42.208269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1292/2562] 10.222.0.45 () {58 vars in 1274 bytes} [Wed May 25 16:46:42 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:46:43.431203 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-9d34b815-794b-46ba-9862-6520f5485c68 req-10f24eef-540d-45e5-b67f-33797150f26f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "54076037-ad26-40b3-9791-13f7b1aa56fd", "tag": "e978b244-ff54-4356-8f47-d64ca37af707"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:46:43.436699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=6 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:46:43.443568 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-9d34b815-794b-46ba-9862-6520f5485c68 req-10f24eef-540d-45e5-b67f-33797150f26f service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:46:43.444099 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-9d34b815-794b-46ba-9862-6520f5485c68 req-10f24eef-540d-45e5-b67f-33797150f26f service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:46:43.455817 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-9d34b815-794b-46ba-9862-6520f5485c68 req-10f24eef-540d-45e5-b67f-33797150f26f service nova] Creating event network-changed:e978b244-ff54-4356-8f47-d64ca37af707 for instance 54076037-ad26-40b3-9791-13f7b1aa56fd on hv-cinder-84327 May 25 16:46:43.460941 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-9d34b815-794b-46ba-9862-6520f5485c68 req-10f24eef-540d-45e5-b67f-33797150f26f service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033775 May 25 16:46:43.461656 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1271/2563] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:46:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:47:19.715949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:47:19.717174 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=6 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:47:19.719637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:47:19.720882 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:47:19.728199 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-fde81489-c6c8-4748-88b8-431ea0557021 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:47:19.729011 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-fde81489-c6c8-4748-88b8-431ea0557021 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:47:19.740519 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fde81489-c6c8-4748-88b8-431ea0557021 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:47:19.741186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fde81489-c6c8-4748-88b8-431ea0557021 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:47:19.742393 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fde81489-c6c8-4748-88b8-431ea0557021 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:47:19.743097 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-fde81489-c6c8-4748-88b8-431ea0557021 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:47:19.768505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-fde81489-c6c8-4748-88b8-431ea0557021 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd/ips" status: 200 len: 150 microversion: 2.1 time: 0.054109 May 25 16:47:19.769082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1293/2564] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:47:19 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd/ips => generated 150 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:47:38.814917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:47:38.814917 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:47:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:48:19.848178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:48:19.849313 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=68,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:48:19.854537 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-3ef2be2f-b5e9-43ce-a5e5-46225f6becfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:19.855123 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3ef2be2f-b5e9-43ce-a5e5-46225f6becfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:19.863910 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3ef2be2f-b5e9-43ce-a5e5-46225f6becfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:19.864441 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3ef2be2f-b5e9-43ce-a5e5-46225f6becfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:19.865198 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3ef2be2f-b5e9-43ce-a5e5-46225f6becfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:19.865631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-3ef2be2f-b5e9-43ce-a5e5-46225f6becfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:19.893037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-3ef2be2f-b5e9-43ce-a5e5-46225f6becfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:48:19.989107 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-3ef2be2f-b5e9-43ce-a5e5-46225f6becfc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 204 len: 0 microversion: 2.1 time: 0.145521 May 25 16:48:19.989979 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1272/2565] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:48:19 2022] DELETE /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:48:19.996736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR oslo.messaging._drivers.impl_rabbit [-] [8fcadda7-7454-4e25-aa8e-2a49c5484764] AMQP server on 10.222.0.45:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: May 25 16:48:19.997273 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:48:19.998104 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=4 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:48:20.002246 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f40b1ce7-648f-42a6-b892-1d5e05e2888c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:20.002850 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f40b1ce7-648f-42a6-b892-1d5e05e2888c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:20.009538 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f40b1ce7-648f-42a6-b892-1d5e05e2888c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:20.009974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f40b1ce7-648f-42a6-b892-1d5e05e2888c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:20.010713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f40b1ce7-648f-42a6-b892-1d5e05e2888c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:20.011137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f40b1ce7-648f-42a6-b892-1d5e05e2888c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:20.148695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f40b1ce7-648f-42a6-b892-1d5e05e2888c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:48:20.156328 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f40b1ce7-648f-42a6-b892-1d5e05e2888c tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1682 microversion: 2.1 time: 0.161905 May 25 16:48:20.156972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1294/2566] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:48:19 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:48:21.179125 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c83b169f-f88d-4c29-8de8-4df210c605ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:21.180357 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c83b169f-f88d-4c29-8de8-4df210c605ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:21.191658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c83b169f-f88d-4c29-8de8-4df210c605ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:21.192532 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c83b169f-f88d-4c29-8de8-4df210c605ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:21.193308 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c83b169f-f88d-4c29-8de8-4df210c605ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:21.193731 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c83b169f-f88d-4c29-8de8-4df210c605ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:21.335529 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c83b169f-f88d-4c29-8de8-4df210c605ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:48:21.343630 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c83b169f-f88d-4c29-8de8-4df210c605ba tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1682 microversion: 2.1 time: 0.171529 May 25 16:48:21.344162 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1273/2567] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:48:21 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1682 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:48:22.299803 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-2c2951cb-25aa-4c03-b678-c394ef5834b5 req-e4350224-d9e3-44cf-9403-ff21bacc02b5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54076037-ad26-40b3-9791-13f7b1aa56fd", "name": "network-vif-deleted", "tag": "e978b244-ff54-4356-8f47-d64ca37af707"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:48:22.309443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-2c2951cb-25aa-4c03-b678-c394ef5834b5 req-e4350224-d9e3-44cf-9403-ff21bacc02b5 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:22.310155 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-2c2951cb-25aa-4c03-b678-c394ef5834b5 req-e4350224-d9e3-44cf-9403-ff21bacc02b5 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:22.325700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-2c2951cb-25aa-4c03-b678-c394ef5834b5 req-e4350224-d9e3-44cf-9403-ff21bacc02b5 service nova] Creating event network-vif-deleted:e978b244-ff54-4356-8f47-d64ca37af707 for instance 54076037-ad26-40b3-9791-13f7b1aa56fd on hv-cinder-84327 May 25 16:48:22.333439 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-2c2951cb-25aa-4c03-b678-c394ef5834b5 req-e4350224-d9e3-44cf-9403-ff21bacc02b5 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036674 May 25 16:48:22.336630 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1295/2568] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:48:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:48:22.361010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-520be3b3-8391-4f7c-9ec3-dd640f7d1c00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:22.361946 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-520be3b3-8391-4f7c-9ec3-dd640f7d1c00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:22.373955 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-520be3b3-8391-4f7c-9ec3-dd640f7d1c00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:22.374402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-520be3b3-8391-4f7c-9ec3-dd640f7d1c00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:22.375202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-520be3b3-8391-4f7c-9ec3-dd640f7d1c00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:22.377092 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-520be3b3-8391-4f7c-9ec3-dd640f7d1c00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:22.457120 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-520be3b3-8391-4f7c-9ec3-dd640f7d1c00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0c9f3f10fd142439521e4b36cf33157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec1d676becac4112a7c0c319e54b2cd2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:48:22.465196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-520be3b3-8391-4f7c-9ec3-dd640f7d1c00 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 200 len: 1326 microversion: 2.1 time: 0.108008 May 25 16:48:22.470178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1274/2569] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:48:22 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 1326 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:48:23.486913 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:23.487954 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:23.491350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] [8fcadda7-7454-4e25-aa8e-2a49c5484764] Reconnected to AMQP server on 10.222.0.45:5672 via [amqp] client with port 51086. May 25 16:48:23.499581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:23.500327 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:23.501570 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:23.502282 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:23.517137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd HTTP/1.1 May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:48:23.518763 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:23.520672 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '54076037-ad26-40b3-9791-13f7b1aa56fd'} May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:48:23.521695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:23.523808 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:23.525320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:48:23.525320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-e827ff4f-70d6-4ab8-a710-cc5672cfdcdc tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 404 len: 111 microversion: 2.1 time: 0.040458 May 25 16:48:23.525320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1296/2570] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:48:23 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:48:23.529082 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-32c4d478-e26a-4d27-9f8a-9d859f5b8d08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '.version_select at 0x7f4a236b14c0>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:23.549248 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-32c4d478-e26a-4d27-9f8a-9d859f5b8d08 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1925197884" status: 202 len: 0 microversion: 2.1 time: 0.023925 May 25 16:48:23.549796 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1275/2571] 10.222.0.45 () {58 vars in 1220 bytes} [Wed May 25 16:48:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1925197884 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:48:24.191624 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:24.192320 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:24.198863 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.199344 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.200103 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.200562 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.214061 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd HTTP/1.1 May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:48:24.215619 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.217367 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '54076037-ad26-40b3-9791-13f7b1aa56fd'} May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:48:24.218389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.220167 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.221847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:48:24.221847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-267ae2d8-f405-4917-b620-e653dbf252c1 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 404 len: 111 microversion: 2.1 time: 0.032062 May 25 16:48:24.221847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1297/2572] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:48:24 2022] DELETE /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:48:24.226761 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:24.227403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 54076037-ad26-40b3-9791-13f7b1aa56fd] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:24.233538 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.233999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.234752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.235211 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.251402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd HTTP/1.1 May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:48:24.253037 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.255002 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '54076037-ad26-40b3-9791-13f7b1aa56fd'} May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:48:24.256051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. May 25 16:48:24.257669 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.259255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 54076037-ad26-40b3-9791-13f7b1aa56fd could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:48:24.259255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e9ec171e-fc09-4516-990f-32f499ea36c9 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd" status: 404 len: 111 microversion: 2.1 time: 0.031963 May 25 16:48:24.259255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1276/2573] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:48:24 2022] GET /compute/v2.1/servers/54076037-ad26-40b3-9791-13f7b1aa56fd => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:48:24.263200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:24.263777 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:24.269870 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.270354 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.271106 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.271574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.285090 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 HTTP/1.1 May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:48:24.286722 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.288822 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '4af3d63d-392d-4abb-bf38-cc4306f5b631'} May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:48:24.289867 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.291554 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.293360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:48:24.293360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-649a598d-28a6-411f-ab7c-d4ef58f7a1a8 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 404 len: 111 microversion: 2.1 time: 0.028885 May 25 16:48:24.293360 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1298/2574] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:48:24 2022] DELETE /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:48:24.296880 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:24.297487 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 4af3d63d-392d-4abb-bf38-cc4306f5b631] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:24.303713 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.304128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.304913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.305346 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.319421 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 HTTP/1.1 May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:48:24.322019 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.323884 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '4af3d63d-392d-4abb-bf38-cc4306f5b631'} May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:48:24.325010 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. May 25 16:48:24.326651 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.328182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 4af3d63d-392d-4abb-bf38-cc4306f5b631 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:48:24.328182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ff36bc33-8c76-4f57-8bad-7988aa6f43a6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631" status: 404 len: 111 microversion: 2.1 time: 0.030948 May 25 16:48:24.328752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1277/2575] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:48:24 2022] GET /compute/v2.1/servers/4af3d63d-392d-4abb-bf38-cc4306f5b631 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:48:24.334966 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:24.335719 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:24.342522 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.342974 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.343778 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.344216 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.358689 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 HTTP/1.1 May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:48:24.360305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.362015 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '667322c4-d9ca-4320-903f-112d84e5f462'} May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:48:24.363029 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.364592 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:48:24.366346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:48:24.366346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-417bb466-779e-466e-b1e5-103e3edb6d45 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 404 len: 111 microversion: 2.1 time: 0.031526 May 25 16:48:24.366346 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1299/2576] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:48:24 2022] DELETE /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:48:24.370291 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:48:24.370939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 667322c4-d9ca-4320-903f-112d84e5f462] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:48:24.377393 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.377851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.378603 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:48:24.379039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:48:24.393459 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 HTTP/1.1 May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:48:24.394999 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.397191 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '667322c4-d9ca-4320-903f-112d84e5f462'} May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:48:24.398262 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. May 25 16:48:24.400380 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:48:24.401862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 667322c4-d9ca-4320-903f-112d84e5f462 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:48:24.401862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0cb314f8-4b3a-4e17-889f-75ae391f99b6 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462" status: 404 len: 111 microversion: 2.1 time: 0.029977 May 25 16:48:24.401862 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1278/2577] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:48:24 2022] GET /compute/v2.1/servers/667322c4-d9ca-4320-903f-112d84e5f462 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:49:03.367751 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:49:03.368663 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:49:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:50:51.084737 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:50:51.085644 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:50:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:50:58.825182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:50:58.826218 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:50:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:51:03.473377 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:51:03.474307 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:51:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:52:13.662918 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:52:13.664114 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:52:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:52:22.910677 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR oslo.messaging._drivers.impl_rabbit [-] [8fcadda7-7454-4e25-aa8e-2a49c5484764] AMQP server on 10.222.0.45:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: May 25 16:52:22.914897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=12 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:52:22.915595 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=16 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:52:22.918549 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR oslo.messaging._drivers.impl_rabbit [-] [dc9d54e5-753d-46ed-8aa1-8535e53878ad] AMQP server on 10.222.0.45:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: May 25 16:52:22.919754 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:52:22.921413 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=32,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:52:22.926145 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-5da1b9ab-9093-4886-88d6-2d0e14d40c12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:22.926648 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5da1b9ab-9093-4886-88d6-2d0e14d40c12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:22.932302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:22.933189 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:22.938147 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5da1b9ab-9093-4886-88d6-2d0e14d40c12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:22.938698 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5da1b9ab-9093-4886-88d6-2d0e14d40c12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:22.939658 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5da1b9ab-9093-4886-88d6-2d0e14d40c12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:22.939957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-5da1b9ab-9093-4886-88d6-2d0e14d40c12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:22.944073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:22.946419 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:22.947874 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:22.948574 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:22.966402 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] HTTP exception thrown: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Request method failure captured: May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a HTTP/1.1 May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaSt3NdfdTff4OH_doadoVd4ihuLN8fcTpnktVD_XWHHevyVxY4tmWr46X1fVz2TuvDT5fSQ1evXFm6-DfkMDQmPYTRLtjUiNx8nVTeCIRoDB7T5LAC8qRTpjGqBOay7ujWQTOdtPe6UQNGpwsHetNPMu5dXLWOI1Kk0Q9IbllFii8 May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:52:22.968066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeTestJSON-755332015 May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeTestJSON-755332015-project May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: e4e4d39c5ed249bc9dc097ba4c24b3d2 May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeTestJSON-755332015-project May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:22.969779 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'f5ce685f-a7e1-41d6-96f4-e5cc19cad00a'} May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:22.970688 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:22.972182 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:22.975651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Returning 404 to user: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:22.975651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-30ec380c-f350-4bbd-ad88-07f63805449e tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "DELETE /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 404 len: 111 microversion: 2.1 time: 0.061933 May 25 16:52:22.975957 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-5da1b9ab-9093-4886-88d6-2d0e14d40c12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:52:22.976150 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1300/2578] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:52:22 2022] DELETE /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:52:22.977947 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:22.978700 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] [instance: f5ce685f-a7e1-41d6-96f4-e5cc19cad00a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:22.987137 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:22.987823 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:22.989820 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:22.990329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:23.006695 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] HTTP exception thrown: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Request method failure captured: May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a HTTP/1.1 May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaSt3NdfdTff4OH_doadoVd4ihuLN8fcTpnktVD_XWHHevyVxY4tmWr46X1fVz2TuvDT5fSQ1evXFm6-DfkMDQmPYTRLtjUiNx8nVTeCIRoDB7T5LAC8qRTpjGqBOay7ujWQTOdtPe6UQNGpwsHetNPMu5dXLWOI1Kk0Q9IbllFii8 May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:52:23.008897 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/b36e59bfe72a49679b3b3657f5a65882"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeTestJSON-755332015 May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: b36e59bfe72a49679b3b3657f5a65882 May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeTestJSON-755332015 May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeTestJSON-755332015-project May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: e4e4d39c5ed249bc9dc097ba4c24b3d2 May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeTestJSON-755332015-project May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:23.011340 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'f5ce685f-a7e1-41d6-96f4-e5cc19cad00a'} May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:23.012606 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. May 25 16:52:23.014074 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:23.015415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] Returning 404 to user: Instance f5ce685f-a7e1-41d6-96f4-e5cc19cad00a could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:23.015415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-86b497df-6a62-4a51-919c-c7e8c6e50abb tempest-AttachVolumeTestJSON-755332015 tempest-AttachVolumeTestJSON-755332015-project] 10.222.0.45 "GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a" status: 404 len: 111 microversion: 2.1 time: 0.036319 May 25 16:52:23.015415 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1301/2579] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:22 2022] GET /compute/v2.1/servers/f5ce685f-a7e1-41d6-96f4-e5cc19cad00a => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:52:23.074202 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-5da1b9ab-9093-4886-88d6-2d0e14d40c12 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 204 len: 0 microversion: 2.1 time: 0.159976 May 25 16:52:23.075106 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1279/2580] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:52:22 2022] DELETE /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:52:23.085456 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-9f165187-42c5-40c5-8129-cf21ef00b435 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:23.086070 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-9f165187-42c5-40c5-8129-cf21ef00b435 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:23.092612 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f165187-42c5-40c5-8129-cf21ef00b435 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:23.093066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f165187-42c5-40c5-8129-cf21ef00b435 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:23.093832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f165187-42c5-40c5-8129-cf21ef00b435 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:23.094275 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-9f165187-42c5-40c5-8129-cf21ef00b435 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:23.366793 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-9f165187-42c5-40c5-8129-cf21ef00b435 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:52:23.375318 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-9f165187-42c5-40c5-8129-cf21ef00b435 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.293676 May 25 16:52:23.375988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1302/2581] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:23 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:23.380014 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:52:23.380156 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:52:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:52:24.394841 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c86d6d0b-ead2-4a5d-9546-08d2b3595093 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:24.396252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c86d6d0b-ead2-4a5d-9546-08d2b3595093 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:24.404717 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c86d6d0b-ead2-4a5d-9546-08d2b3595093 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:24.405458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c86d6d0b-ead2-4a5d-9546-08d2b3595093 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:24.406215 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c86d6d0b-ead2-4a5d-9546-08d2b3595093 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:24.406640 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c86d6d0b-ead2-4a5d-9546-08d2b3595093 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:24.427578 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] [dc9d54e5-753d-46ed-8aa1-8535e53878ad] Reconnected to AMQP server on 10.222.0.45:5672 via [amqp] client with port 51284. May 25 16:52:24.750154 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-c86d6d0b-ead2-4a5d-9546-08d2b3595093 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:52:24.760321 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c86d6d0b-ead2-4a5d-9546-08d2b3595093 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1736 microversion: 2.1 time: 0.371025 May 25 16:52:24.763681 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1280/2582] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:24 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1736 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:24.881083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] [8fcadda7-7454-4e25-aa8e-2a49c5484764] Reconnected to AMQP server on 10.222.0.45:5672 via [amqp] client with port 51294. May 25 16:52:24.973772 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-1756bc75-0ac6-40d9-a394-bdc8160c47a6 req-667a0f65-7e0c-4cac-8c92-5c4f9bccfa5e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d86354dc-f3d4-409d-b331-9f708947be2a", "name": "network-vif-deleted", "tag": "e2c0938e-f9fb-4796-87fa-103cbcfd2536"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:52:24.985487 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-1756bc75-0ac6-40d9-a394-bdc8160c47a6 req-667a0f65-7e0c-4cac-8c92-5c4f9bccfa5e service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:24.985934 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-1756bc75-0ac6-40d9-a394-bdc8160c47a6 req-667a0f65-7e0c-4cac-8c92-5c4f9bccfa5e service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:25.017484 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-1756bc75-0ac6-40d9-a394-bdc8160c47a6 req-667a0f65-7e0c-4cac-8c92-5c4f9bccfa5e service nova] Creating event network-vif-deleted:e2c0938e-f9fb-4796-87fa-103cbcfd2536 for instance d86354dc-f3d4-409d-b331-9f708947be2a on hv-cinder-84327 May 25 16:52:25.024224 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-1756bc75-0ac6-40d9-a394-bdc8160c47a6 req-667a0f65-7e0c-4cac-8c92-5c4f9bccfa5e service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.155235 May 25 16:52:25.024971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1303/2583] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:52:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:25.780687 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2c0b9794-1904-487f-8000-18551f3114cc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:25.782097 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-2c0b9794-1904-487f-8000-18551f3114cc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:25.795208 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c0b9794-1904-487f-8000-18551f3114cc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:25.795954 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c0b9794-1904-487f-8000-18551f3114cc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:25.796998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c0b9794-1904-487f-8000-18551f3114cc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:25.798051 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-2c0b9794-1904-487f-8000-18551f3114cc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:25.932186 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-344e9f30-68be-4f14-af20-31e92c8d356e req-ae377e20-1aa4-46b0-8987-8e90d0043dd9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3fc5dcaa-3275-47eb-88ec-62b23505c91b", "name": "network-vif-deleted", "tag": "e6435373-0460-4e82-8377-2ccbc0dfa40a"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:52:25.948784 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-344e9f30-68be-4f14-af20-31e92c8d356e req-ae377e20-1aa4-46b0-8987-8e90d0043dd9 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:25.949629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-344e9f30-68be-4f14-af20-31e92c8d356e req-ae377e20-1aa4-46b0-8987-8e90d0043dd9 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:25.983524 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-344e9f30-68be-4f14-af20-31e92c8d356e req-ae377e20-1aa4-46b0-8987-8e90d0043dd9 service nova] Creating event network-vif-deleted:e6435373-0460-4e82-8377-2ccbc0dfa40a for instance 3fc5dcaa-3275-47eb-88ec-62b23505c91b on hv-cinder-84327 May 25 16:52:25.988505 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-344e9f30-68be-4f14-af20-31e92c8d356e req-ae377e20-1aa4-46b0-8987-8e90d0043dd9 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.182288 May 25 16:52:25.990310 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1304/2584] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:52:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:52:26.146030 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-2c0b9794-1904-487f-8000-18551f3114cc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:52:26.153897 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2c0b9794-1904-487f-8000-18551f3114cc tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1690 microversion: 2.1 time: 0.377229 May 25 16:52:26.160923 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1281/2585] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:25 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1690 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:27.174021 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-f31c65ed-ff5b-4a1f-9ef7-f0830e43515e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:27.174731 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-f31c65ed-ff5b-4a1f-9ef7-f0830e43515e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:27.183083 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f31c65ed-ff5b-4a1f-9ef7-f0830e43515e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:27.183967 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f31c65ed-ff5b-4a1f-9ef7-f0830e43515e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:27.186347 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f31c65ed-ff5b-4a1f-9ef7-f0830e43515e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:27.187171 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-f31c65ed-ff5b-4a1f-9ef7-f0830e43515e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:27.322064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-f31c65ed-ff5b-4a1f-9ef7-f0830e43515e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:52:27.332742 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-f31c65ed-ff5b-4a1f-9ef7-f0830e43515e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 200 len: 1622 microversion: 2.1 time: 0.164636 May 25 16:52:27.335577 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1305/2586] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:27 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 1622 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:27.348685 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-bc9761f8-390b-4769-a1df-2e45c85cbccf req-99dad3cb-a05f-4c81-9f34-4cdc9d4811f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "621e54b1-cdde-493f-9bad-1c1222a0731a", "name": "network-vif-deleted", "tag": "9b36da50-ea60-4505-bbbd-48a647e0698b"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:52:27.370427 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-bc9761f8-390b-4769-a1df-2e45c85cbccf req-99dad3cb-a05f-4c81-9f34-4cdc9d4811f7 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:27.370827 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-bc9761f8-390b-4769-a1df-2e45c85cbccf req-99dad3cb-a05f-4c81-9f34-4cdc9d4811f7 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:27.391865 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-bc9761f8-390b-4769-a1df-2e45c85cbccf req-99dad3cb-a05f-4c81-9f34-4cdc9d4811f7 service nova] Creating event network-vif-deleted:9b36da50-ea60-4505-bbbd-48a647e0698b for instance 621e54b1-cdde-493f-9bad-1c1222a0731a on hv-cinder-84327 May 25 16:52:27.397769 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-bc9761f8-390b-4769-a1df-2e45c85cbccf req-99dad3cb-a05f-4c81-9f34-4cdc9d4811f7 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053416 May 25 16:52:27.398404 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1282/2587] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:52:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:28.350608 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:28.351263 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 621e54b1-cdde-493f-9bad-1c1222a0731a] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:28.361016 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:28.361464 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:28.362244 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:28.362678 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:28.394757 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a could not be found. May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a HTTP/1.1 May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:52:28.396575 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a could not be found. May 25 16:52:28.398401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '621e54b1-cdde-493f-9bad-1c1222a0731a'} May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a could not be found. May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:28.399597 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a could not be found. May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a could not be found. May 25 16:52:28.401339 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:28.402623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 404 to user: Instance 621e54b1-cdde-493f-9bad-1c1222a0731a could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:28.402623 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-5560f2bd-8688-4763-a4fc-f0952828c063 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a" status: 404 len: 111 microversion: 2.1 time: 0.053834 May 25 16:52:28.404573 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1306/2588] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:28 2022] GET /compute/v2.1/servers/621e54b1-cdde-493f-9bad-1c1222a0731a => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:52:28.412700 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-069c6f2c-7ce0-4666-a3f2-e19a53faf97e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '.version_select at 0x7f4a300215e0>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:28.435432 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-069c6f2c-7ce0-4666-a3f2-e19a53faf97e tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1809206957" status: 202 len: 0 microversion: 2.1 time: 0.029217 May 25 16:52:28.436559 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1283/2589] 10.222.0.45 () {58 vars in 1220 bytes} [Wed May 25 16:52:28 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1809206957 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:52:28.840423 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c47a52d2-b5b2-4a06-bbc6-8864b8c47442 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] Action: 'create', calling method: .version_select at 0x7f4a23e79550>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-625303161", "metadata": {"Type": "work"}}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:52:29.202073 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c47a52d2-b5b2-4a06-bbc6-8864b8c47442 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] 10.222.0.45 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 0.503514 May 25 16:52:29.203195 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1307/2590] 10.222.0.45 () {60 vars in 1155 bytes} [Wed May 25 16:52:28 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 506 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 25 16:52:29.215654 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fd7477d7-b2fe-4a35-a3dc-86f63719bdba tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] Calling method '.version_select at 0x7f4a300e7e50>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:29.288959 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fd7477d7-b2fe-4a35-a3dc-86f63719bdba tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b" status: 200 len: 352 microversion: 2.1 time: 0.078373 May 25 16:52:29.289735 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1284/2591] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:29 2022] GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b => generated 352 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:30.306760 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-8a63e4b4-6a23-4048-ab48-41b9ee425bf8 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] Calling method '.version_select at 0x7f4a351639d0>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:30.358886 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-8a63e4b4-6a23-4048-ab48-41b9ee425bf8 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b" status: 200 len: 353 microversion: 2.1 time: 0.059327 May 25 16:52:30.359488 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1308/2592] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:30 2022] GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b => generated 353 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:30.370848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-25ad0296-bce3-44d4-9cc8-9ca99b0aa687 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] Calling method '.version_select at 0x7f4a236fd430>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:30.438991 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-25ad0296-bce3-44d4-9cc8-9ca99b0aa687 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b" status: 200 len: 353 microversion: 2.1 time: 0.072432 May 25 16:52:30.439683 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1285/2593] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:30 2022] GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b => generated 353 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:52:30.455691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7ff6bdae-e94f-498c-9430-4b786cabc450 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] Calling method '.version_select at 0x7f4a351b0160>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:30.521725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7ff6bdae-e94f-498c-9430-4b786cabc450 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] 10.222.0.45 "DELETE /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b" status: 202 len: 0 microversion: 2.1 time: 0.069564 May 25 16:52:30.522283 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1309/2594] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:52:30 2022] DELETE /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b => generated 0 bytes in 71 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:52:30.531036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-ef70c11a-0b55-4483-912f-621f0c43385a tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] Calling method '.version_select at 0x7f4a236fdee0>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:30.581420 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-ef70c11a-0b55-4483-912f-621f0c43385a tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b" status: 200 len: 352 microversion: 2.1 time: 0.054372 May 25 16:52:30.581989 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1286/2595] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:30 2022] GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b => generated 352 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:52:31.578300 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-16d4f45d-1053-4c39-a7f5-713eb6ca2b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:31.578829 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-16d4f45d-1053-4c39-a7f5-713eb6ca2b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:31.585389 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-16d4f45d-1053-4c39-a7f5-713eb6ca2b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:31.585844 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-16d4f45d-1053-4c39-a7f5-713eb6ca2b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:31.586600 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-16d4f45d-1053-4c39-a7f5-713eb6ca2b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:31.587034 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-16d4f45d-1053-4c39-a7f5-713eb6ca2b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:31.594252 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2b31db50-3f3a-4b65-a639-cf3fd1727412 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] Calling method '.version_select at 0x7f4a30030550>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:31.613612 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-16d4f45d-1053-4c39-a7f5-713eb6ca2b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Going to try to terminate instance {{(pid=117795) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:52:31.622699 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-2b31db50-3f3a-4b65-a639-cf3fd1727412 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] HTTP exception thrown: Volume f924fe21-8de9-45b6-8c0f-4b7f1521b83b could not be found. May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2b31db50-3f3a-4b65-a639-cf3fd1727412 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] Request method failure captured: May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b HTTP/1.1 May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijl7MUQDSAZUFIcaDxOGP5J485Qa3EFxYfzamS1fMyyv868TUB3jEc0jH8GdXkCuAGc-KnKNqrs4fX9fpJo0WsS8hku89pBOvaZELUcFrA3g9gzPPkuJrFNO4Oqe0q54f5dASVhZwkpWbcf6vNoUvZi_2uGPFc5URmVyiABTGVWR_ytQ May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: eed6bcf1b81541df9b1476c021f5c220 May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesGetTestJSON-1133729599 May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:52:31.624911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_eed6bcf1b81541df9b1476c021f5c220"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/eed6bcf1b81541df9b1476c021f5c220"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/eed6bcf1b81541df9b1476c021f5c220"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/eed6bcf1b81541df9b1476c021f5c220"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesGetTestJSON-1133729599 May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: eed6bcf1b81541df9b1476c021f5c220 May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesGetTestJSON-1133729599 May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesGetTestJSON-1133729599-project May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: c9f67808fe2440cd9d92dd3b6d0f2ff9 May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesGetTestJSON-1133729599-project May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Volume f924fe21-8de9-45b6-8c0f-4b7f1521b83b could not be found. May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'f924fe21-8de9-45b6-8c0f-4b7f1521b83b'} May 25 16:52:31.626715 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi item = cinderclient( May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume f924fe21-8de9-45b6-8c0f-4b7f1521b83b could not be found. (HTTP 404) (Request-ID: req-3fcd691e-fe65-464e-a70c-dd61f6d792d5) May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:31.629039 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi item = cinderclient( May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:52:31.630690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume f924fe21-8de9-45b6-8c0f-4b7f1521b83b could not be found. May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 25 16:52:31.632343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume f924fe21-8de9-45b6-8c0f-4b7f1521b83b could not be found. May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume f924fe21-8de9-45b6-8c0f-4b7f1521b83b could not be found. May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-2b31db50-3f3a-4b65-a639-cf3fd1727412 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] Returning 404 to user: Volume f924fe21-8de9-45b6-8c0f-4b7f1521b83b could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-2b31db50-3f3a-4b65-a639-cf3fd1727412 tempest-VolumesGetTestJSON-1133729599 tempest-VolumesGetTestJSON-1133729599-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b" status: 404 len: 109 microversion: 2.1 time: 0.036418 May 25 16:52:31.633913 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1287/2596] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:31 2022] GET /compute/v2.1/os-volumes/f924fe21-8de9-45b6-8c0f-4b7f1521b83b => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:52:31.706666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-16d4f45d-1053-4c39-a7f5-713eb6ca2b56 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 204 len: 0 microversion: 2.1 time: 0.132356 May 25 16:52:31.707205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1310/2597] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:52:31 2022] DELETE /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:52:31.718939 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-9ff01288-1c28-46e8-9edb-7f8f9236f92c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:31.719675 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-9ff01288-1c28-46e8-9edb-7f8f9236f92c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:31.726115 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ff01288-1c28-46e8-9edb-7f8f9236f92c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:31.726561 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ff01288-1c28-46e8-9edb-7f8f9236f92c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:31.727340 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ff01288-1c28-46e8-9edb-7f8f9236f92c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:31.727765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-9ff01288-1c28-46e8-9edb-7f8f9236f92c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:31.866934 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-9ff01288-1c28-46e8-9edb-7f8f9236f92c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:52:31.874876 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-9ff01288-1c28-46e8-9edb-7f8f9236f92c tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1508 microversion: 2.1 time: 0.162810 May 25 16:52:31.875453 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1288/2598] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:31 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1508 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:32.776503 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-b96e643a-1726-4e73-b487-20843fd1ce54 req-a92f0a09-8f5d-4e15-b4ab-3cac708af8bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f858e1ab-1766-4273-be90-d3c73f8678da", "name": "network-vif-deleted", "tag": "dc3d60cd-b59c-445a-84d9-b0658a3a8344"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:52:32.786143 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-b96e643a-1726-4e73-b487-20843fd1ce54 req-a92f0a09-8f5d-4e15-b4ab-3cac708af8bf service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:32.786964 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-b96e643a-1726-4e73-b487-20843fd1ce54 req-a92f0a09-8f5d-4e15-b4ab-3cac708af8bf service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:32.799736 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-b96e643a-1726-4e73-b487-20843fd1ce54 req-a92f0a09-8f5d-4e15-b4ab-3cac708af8bf service nova] Creating event network-vif-deleted:dc3d60cd-b59c-445a-84d9-b0658a3a8344 for instance f858e1ab-1766-4273-be90-d3c73f8678da on hv-cinder-84327 May 25 16:52:32.804890 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-b96e643a-1726-4e73-b487-20843fd1ce54 req-a92f0a09-8f5d-4e15-b4ab-3cac708af8bf service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032366 May 25 16:52:32.805816 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1311/2599] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:52:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:32.893881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:32.894516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:32.901591 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:32.902047 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:32.902806 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:32.903267 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:32.935314 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:32.935790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:32.936302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.objects.instance [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lazy-loading 'fault' on Instance uuid f858e1ab-1766-4273-be90-d3c73f8678da {{(pid=117794) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:52:32.983871 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:52:32.991790 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-130b6477-38af-4f95-bd5e-b17a754865b2 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 200 len: 1312 microversion: 2.1 time: 0.102447 May 25 16:52:32.992475 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1289/2600] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:32 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 1312 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:34.009714 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:34.011013 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: f858e1ab-1766-4273-be90-d3c73f8678da] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:34.017984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:34.018430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:34.019222 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:34.019655 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:34.035062 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Instance f858e1ab-1766-4273-be90-d3c73f8678da could not be found. May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da HTTP/1.1 May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:52:34.036725 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance f858e1ab-1766-4273-be90-d3c73f8678da could not be found. May 25 16:52:34.038817 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'f858e1ab-1766-4273-be90-d3c73f8678da'} May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f858e1ab-1766-4273-be90-d3c73f8678da could not be found. May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:34.040601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f858e1ab-1766-4273-be90-d3c73f8678da could not be found. May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f858e1ab-1766-4273-be90-d3c73f8678da could not be found. May 25 16:52:34.042991 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:34.044946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 404 to user: Instance f858e1ab-1766-4273-be90-d3c73f8678da could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:34.044946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6c46fc66-38b0-4df7-b7a1-d8d9c1ecc04f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da" status: 404 len: 111 microversion: 2.1 time: 0.033700 May 25 16:52:34.044946 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1312/2601] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:34 2022] GET /compute/v2.1/servers/f858e1ab-1766-4273-be90-d3c73f8678da => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:52:34.048504 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c4aeeb72-378f-4040-aa74-ea596b72d8e3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:34.049160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c4aeeb72-378f-4040-aa74-ea596b72d8e3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:34.056417 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4aeeb72-378f-4040-aa74-ea596b72d8e3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:34.057160 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4aeeb72-378f-4040-aa74-ea596b72d8e3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:34.057951 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4aeeb72-378f-4040-aa74-ea596b72d8e3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:34.058417 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c4aeeb72-378f-4040-aa74-ea596b72d8e3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:34.085703 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c4aeeb72-378f-4040-aa74-ea596b72d8e3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Going to try to terminate instance {{(pid=117794) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:52:34.167484 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c4aeeb72-378f-4040-aa74-ea596b72d8e3 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "DELETE /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 204 len: 0 microversion: 2.1 time: 0.123605 May 25 16:52:34.168366 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1290/2602] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:52:34 2022] DELETE /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:52:34.176673 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-850443d0-3342-4463-9859-a6aaa7a31820 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:34.177267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-850443d0-3342-4463-9859-a6aaa7a31820 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:34.183269 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-850443d0-3342-4463-9859-a6aaa7a31820 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:34.183712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-850443d0-3342-4463-9859-a6aaa7a31820 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:34.184449 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-850443d0-3342-4463-9859-a6aaa7a31820 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:34.184876 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-850443d0-3342-4463-9859-a6aaa7a31820 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:34.314098 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-850443d0-3342-4463-9859-a6aaa7a31820 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:52:34.321845 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-850443d0-3342-4463-9859-a6aaa7a31820 tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1520 microversion: 2.1 time: 0.148987 May 25 16:52:34.322308 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1313/2603] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:34 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1520 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:52:35.058523 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-29e29fd0-3024-4b26-b67c-1bc2de6d34d8 req-ec0ecdf2-356e-493b-8a9d-dea535efaaaa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69fc8c5b-82f0-4a03-8dab-381fdee5884d", "name": "network-vif-deleted", "tag": "4220f323-95a7-4c14-8c3e-42c4591d25b3"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:52:35.068175 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-29e29fd0-3024-4b26-b67c-1bc2de6d34d8 req-ec0ecdf2-356e-493b-8a9d-dea535efaaaa service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:35.068632 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-29e29fd0-3024-4b26-b67c-1bc2de6d34d8 req-ec0ecdf2-356e-493b-8a9d-dea535efaaaa service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:35.085780 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-29e29fd0-3024-4b26-b67c-1bc2de6d34d8 req-ec0ecdf2-356e-493b-8a9d-dea535efaaaa service nova] Creating event network-vif-deleted:4220f323-95a7-4c14-8c3e-42c4591d25b3 for instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d on hv-cinder-84327 May 25 16:52:35.090566 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-29e29fd0-3024-4b26-b67c-1bc2de6d34d8 req-ec0ecdf2-356e-493b-8a9d-dea535efaaaa service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036071 May 25 16:52:35.091182 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1291/2604] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:52:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:35.340284 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:35.340905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:35.350770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:35.350975 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:35.353815 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:35.354257 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:35.401972 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:35.402498 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:35.402960 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lazy-loading 'fault' on Instance uuid 69fc8c5b-82f0-4a03-8dab-381fdee5884d {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:52:35.482084 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '829c30cd914f48548b987bd6e39ee774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd0c9fdb2e8e4e72ac2835b0482ae824', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:52:35.495039 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-75c68c63-8061-43bb-bf7f-66d9156185cd tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 200 len: 1311 microversion: 2.1 time: 0.159584 May 25 16:52:35.496363 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1314/2605] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:35 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 1311 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:36.516895 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:36.517631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] [instance: 69fc8c5b-82f0-4a03-8dab-381fdee5884d] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:36.525975 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:36.526552 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:36.527608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:36.528172 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:36.544848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] HTTP exception thrown: Instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d could not be found. May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Request method failure captured: May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d HTTP/1.1 May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaaoEaPDjR1WJcMfs25lGnYxEsQXgix9ZptFd1xBVThHZf8qaB4KawMwloi62ZBLNGxlYjezmxnInuzRvLRKoyg_lzlwfCRi8fRR57_Md9pNdDcKmAQcEWjHxPtBLKDe78hv5yzf0tGjt_mMUldCR_0XQa82SbfEXCb6WD-jG4wHn4 May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:52:36.546652 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/bd0c9fdb2e8e4e72ac2835b0482ae824"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: bd0c9fdb2e8e4e72ac2835b0482ae824 May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-781582452 May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 829c30cd914f48548b987bd6e39ee774 May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-ServerRescueNegativeTestJSON-781582452-project May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d could not be found. May 25 16:52:36.548246 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '69fc8c5b-82f0-4a03-8dab-381fdee5884d'} May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d could not be found. May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:36.549151 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d could not be found. May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d could not be found. May 25 16:52:36.550564 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:36.553031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] Returning 404 to user: Instance 69fc8c5b-82f0-4a03-8dab-381fdee5884d could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:36.553031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c772eb0b-60cf-4816-b3c0-13c0507f594f tempest-ServerRescueNegativeTestJSON-781582452 tempest-ServerRescueNegativeTestJSON-781582452-project] 10.222.0.45 "GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d" status: 404 len: 111 microversion: 2.1 time: 0.038650 May 25 16:52:36.553031 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1292/2606] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:52:36 2022] GET /compute/v2.1/servers/69fc8c5b-82f0-4a03-8dab-381fdee5884d => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:52:36.799648 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:36.800134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:36.807018 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:36.807499 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:36.808255 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:36.808680 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:36.989381 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] HTTP exception thrown: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Request method failure captured: May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 HTTP/1.1 May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaS4R2Q0uyyQiC_zqcGI-JA7bCVvbPwrz_330b_JgHbpSrVu9wM-WpOvctaejhwGxpAodwx3WJWsuQ0GFkUbHXQCzUkq6Fe6YZKZ15x3DYZvoWLLf8QyMPtbUZHDXgGJ7Z2fImgS--Ix-P6gbsMoij8yzFICYuR4jTIofo8HNi4SqA May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Openstack-Nova-Api-Version: 2.20 May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:52:36.990969 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: 06ba885a0b164134898c42a332feba76 May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:36.992909 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4'} May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:36.993978 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:36.995750 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:36.997584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Returning 404 to user: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:36.997584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-09df1b66-dfdd-4602-9cf5-e7a7bbb46bcd tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "DELETE /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 404 len: 111 microversion: 2.20 time: 0.196751 May 25 16:52:36.997584 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1315/2607] 10.222.0.45 () {60 vars in 1279 bytes} [Wed May 25 16:52:36 2022] DELETE /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 111 bytes in 198 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 25 16:52:37.001385 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:37.001988 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] [instance: 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:52:37.008416 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:37.008881 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:37.009641 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:52:37.010062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:52:37.025490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] HTTP exception thrown: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Request method failure captured: May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 HTTP/1.1 May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaS4R2Q0uyyQiC_zqcGI-JA7bCVvbPwrz_330b_JgHbpSrVu9wM-WpOvctaejhwGxpAodwx3WJWsuQ0GFkUbHXQCzUkq6Fe6YZKZ15x3DYZvoWLLf8QyMPtbUZHDXgGJ7Z2fImgS--Ix-P6gbsMoij8yzFICYuR4jTIofo8HNi4SqA May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Openstack-Nova-Api-Version: 2.20 May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:52:37.027074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/c3003077416c4ce997e3aa7e47e8e3b7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: c3003077416c4ce997e3aa7e47e8e3b7 May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-109583211 May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: 06ba885a0b164134898c42a332feba76 May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeShelveTestJSON-109583211-project May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:37.029474 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4'} May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:37.030436 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. May 25 16:52:37.031966 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:37.033255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] Returning 404 to user: Instance 9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:37.033255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-215884de-9124-4462-b23c-1df9345dcb9e tempest-AttachVolumeShelveTestJSON-109583211 tempest-AttachVolumeShelveTestJSON-109583211-project] 10.222.0.45 "GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4" status: 404 len: 111 microversion: 2.20 time: 0.031097 May 25 16:52:37.033255 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1293/2608] 10.222.0.45 () {60 vars in 1276 bytes} [Wed May 25 16:52:36 2022] GET /compute/v2.1/servers/9f1c6d00-eaad-4aaa-abae-56ea6ea2d8b4 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 25 16:52:42.556264 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-deb706b8-b1d1-424b-9a6e-e779f7baa8b5 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Action: 'create', calling method: .version_select at 0x7f4a351b03a0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1260618913"}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:52:42.868102 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-deb706b8-b1d1-424b-9a6e-e779f7baa8b5 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 0.497066 May 25 16:52:42.868780 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1316/2609] 10.222.0.45 () {60 vars in 1154 bytes} [Wed May 25 16:52:42 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 498 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 25 16:52:42.879579 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0ba0b978-e92d-446e-b35d-83ad5323a534 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a237f9d30>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:42.928631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0ba0b978-e92d-446e-b35d-83ad5323a534 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588" status: 200 len: 345 microversion: 2.1 time: 0.052808 May 25 16:52:42.929196 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1294/2610] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:42 2022] GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 => generated 345 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:52:43.597365 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-eaa09ad1-e321-4b69-a9c7-de3db20d2515 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a351b0280>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:43.647766 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-eaa09ad1-e321-4b69-a9c7-de3db20d2515 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/1215b2d7-8d85-418c-87eb-cd89522f95ba" status: 200 len: 351 microversion: 2.1 time: 0.054702 May 25 16:52:43.649226 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1317/2611] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:43 2022] GET /compute/v2.1/os-volumes/1215b2d7-8d85-418c-87eb-cd89522f95ba => generated 351 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:43.941768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e3c41e8b-6a32-4fab-85a5-a6c836a9f5bf tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a236ebdc0>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:43.990940 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e3c41e8b-6a32-4fab-85a5-a6c836a9f5bf tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588" status: 200 len: 346 microversion: 2.1 time: 0.053535 May 25 16:52:43.991540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1295/2612] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:43 2022] GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 => generated 346 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:44.002646 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-51fc1782-3747-40d6-83da-b837ba968e13 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Action: 'create', calling method: .version_select at 0x7f4a351a54c0>, body: {"snapshot": {"volume_id": "bfb8f712-4fb4-48dd-a53b-615bd89d4588", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-63670780"}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:52:44.169279 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-51fc1782-3747-40d6-83da-b837ba968e13 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "POST /compute/v2.1/os-snapshots" status: 200 len: 285 microversion: 2.1 time: 0.170219 May 25 16:52:44.169912 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1318/2613] 10.222.0.45 () {60 vars in 1161 bytes} [Wed May 25 16:52:43 2022] POST /compute/v2.1/os-snapshots => generated 285 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:44.182403 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0de0af94-3469-4e11-9858-860cb28d67af tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a237c01f0>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:44.213495 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0de0af94-3469-4e11-9858-860cb28d67af tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54" status: 200 len: 285 microversion: 2.1 time: 0.035706 May 25 16:52:44.214348 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1296/2614] 10.222.0.45 () {58 vars in 1250 bytes} [Wed May 25 16:52:44 2022] GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54 => generated 285 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:52:44.224411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-c42a8c13-d923-4ca2-b3a0-9f32e7cd9ded tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a30cc2940>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:44.257119 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-c42a8c13-d923-4ca2-b3a0-9f32e7cd9ded tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-snapshots" status: 200 len: 288 microversion: 2.1 time: 0.036134 May 25 16:52:44.257848 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1319/2615] 10.222.0.45 () {58 vars in 1139 bytes} [Wed May 25 16:52:44 2022] GET /compute/v2.1/os-snapshots => generated 288 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:44.272825 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-bac44367-df71-4afc-869b-d3842869917e tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a237f9a60>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:44.304741 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-bac44367-df71-4afc-869b-d3842869917e tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54" status: 200 len: 285 microversion: 2.1 time: 0.037825 May 25 16:52:44.305299 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1297/2616] 10.222.0.45 () {58 vars in 1250 bytes} [Wed May 25 16:52:44 2022] GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54 => generated 285 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:44.991691 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b059c08e-e3b9-4e65-bbb1-e3293dc36d85 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a30d10280>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:45.039841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b059c08e-e3b9-4e65-bbb1-e3293dc36d85 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/e1267d6a-26e9-4ef7-84b2-1eb8ef8a2a60" status: 200 len: 351 microversion: 2.1 time: 0.052392 May 25 16:52:45.040399 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1320/2617] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:44 2022] GET /compute/v2.1/os-volumes/e1267d6a-26e9-4ef7-84b2-1eb8ef8a2a60 => generated 351 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:45.316512 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-d7ce6e4a-935f-4dbe-97cd-9694686d2df1 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a236fd790>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:45.372022 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-d7ce6e4a-935f-4dbe-97cd-9694686d2df1 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54" status: 200 len: 285 microversion: 2.1 time: 0.059903 May 25 16:52:45.372577 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1298/2618] 10.222.0.45 () {58 vars in 1250 bytes} [Wed May 25 16:52:45 2022] GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54 => generated 285 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:46.383637 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1beafa7a-cba2-4332-9694-8b4ecbcbcd68 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a351a5820>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:46.415316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1beafa7a-cba2-4332-9694-8b4ecbcbcd68 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54" status: 200 len: 285 microversion: 2.1 time: 0.035149 May 25 16:52:46.416040 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1321/2619] 10.222.0.45 () {58 vars in 1250 bytes} [Wed May 25 16:52:46 2022] GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54 => generated 285 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:47.421183 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:52:47.422711 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell1:SELECT=62,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:52:47.423656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:52:47.424527 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:52:47.428206 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-e79a6735-3a01-4063-9f6e-c3b3f11d4837 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a300fab80>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:47.429532 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4d9a5b19-9db2-4cd3-8f57-83d053e7657e tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a351b0ee0>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:47.497257 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-e79a6735-3a01-4063-9f6e-c3b3f11d4837 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/ec351cc1-73c2-4495-98ba-69bfeba6e148" status: 200 len: 351 microversion: 2.1 time: 0.077494 May 25 16:52:47.497832 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1299/2620] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:47 2022] GET /compute/v2.1/os-volumes/ec351cc1-73c2-4495-98ba-69bfeba6e148 => generated 351 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:47.502350 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4d9a5b19-9db2-4cd3-8f57-83d053e7657e tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54" status: 200 len: 286 microversion: 2.1 time: 0.079425 May 25 16:52:47.503267 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1322/2621] 10.222.0.45 () {58 vars in 1250 bytes} [Wed May 25 16:52:47 2022] GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54 => generated 286 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:47.509071 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-12259c43-16ac-4f1c-9a4a-c692a3082d2d tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a237c0430>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:47.513450 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1536a28e-3a49-46a6-8898-5391ee047e9f tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a30cc2b80>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:47.579943 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1536a28e-3a49-46a6-8898-5391ee047e9f tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "DELETE /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54" status: 202 len: 0 microversion: 2.1 time: 0.070522 May 25 16:52:47.580556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1323/2622] 10.222.0.45 () {58 vars in 1253 bytes} [Wed May 25 16:52:47 2022] DELETE /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54 => generated 0 bytes in 72 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 25 16:52:47.588786 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-62a5c976-36d5-4731-a840-feec04858827 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a23e449d0>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:47.601998 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-12259c43-16ac-4f1c-9a4a-c692a3082d2d tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.096649 May 25 16:52:47.602553 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1300/2623] 10.222.0.45 () {58 vars in 1133 bytes} [Wed May 25 16:52:47 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:52:47.614479 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-c61b8f53-d4b3-421d-96a7-c116e853363c tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a237e9f70>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:47.618531 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-62a5c976-36d5-4731-a840-feec04858827 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54" status: 200 len: 285 microversion: 2.1 time: 0.033360 May 25 16:52:47.619134 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1324/2624] 10.222.0.45 () {58 vars in 1250 bytes} [Wed May 25 16:52:47 2022] GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54 => generated 285 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 25 16:52:47.696496 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-c61b8f53-d4b3-421d-96a7-c116e853363c tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.085558 May 25 16:52:47.697074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1301/2625] 10.222.0.45 () {58 vars in 1148 bytes} [Wed May 25 16:52:47 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:47.709302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-90cb5955-05f8-4978-a6bb-4ccbfaad2298 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a351a5ee0>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:47.790703 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-90cb5955-05f8-4978-a6bb-4ccbfaad2298 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.085301 May 25 16:52:47.791302 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1325/2626] 10.222.0.45 () {58 vars in 1133 bytes} [Wed May 25 16:52:47 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:47.802325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-46c545eb-d0db-4eac-8f5e-8133b9f9baa2 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a301ad670>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:47.885765 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-46c545eb-d0db-4eac-8f5e-8133b9f9baa2 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.087070 May 25 16:52:47.886325 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1302/2627] 10.222.0.45 () {58 vars in 1166 bytes} [Wed May 25 16:52:47 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:47.902023 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-4d504dd0-6470-49d6-b7a2-d38413100c8d tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a351a5af0>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:47.987472 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-4d504dd0-6470-49d6-b7a2-d38413100c8d tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.092328 May 25 16:52:47.988028 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1326/2628] 10.222.0.45 () {58 vars in 1169 bytes} [Wed May 25 16:52:47 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:47.999766 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-a93182d3-28bb-437a-b5ad-a95e85af62f1 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a30021430>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:48.081311 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-a93182d3-28bb-437a-b5ad-a95e85af62f1 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.085199 May 25 16:52:48.081879 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1303/2629] 10.222.0.45 () {58 vars in 1154 bytes} [Wed May 25 16:52:47 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:48.095670 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-7722f9d6-b58d-486f-b12c-36f9e9f10f40 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a30cc2af0>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:48.180999 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-7722f9d6-b58d-486f-b12c-36f9e9f10f40 tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.091374 May 25 16:52:48.181656 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1327/2630] 10.222.0.45 () {58 vars in 1187 bytes} [Wed May 25 16:52:48 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:48.193343 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f0bf7f89-76d7-4ea7-a1a5-65eab8c30d1f tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] Calling method '.version_select at 0x7f4a30030700>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:48.278787 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f0bf7f89-76d7-4ea7-a1a5-65eab8c30d1f tempest-VolumesTestJSON-1333669254 tempest-VolumesTestJSON-1333669254-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.088886 May 25 16:52:48.279746 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1304/2631] 10.222.0.45 () {58 vars in 1154 bytes} [Wed May 25 16:52:48 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:52:48.630502 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b5825320-ebcd-4f2a-8b02-59528dbbb93e tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a23fbb550>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:48.656046 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-b5825320-ebcd-4f2a-8b02-59528dbbb93e tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] HTTP exception thrown: Snapshot f442de9d-9b99-4cbb-b2a1-19513859aa54 could not be found. May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b5825320-ebcd-4f2a-8b02-59528dbbb93e tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Request method failure captured: May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54 HTTP/1.1 May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijl7a60VPtxCXOQpP5Bw_iAG4nNiQPVedCqsBM0LZCksk2u4Kt1rS5juOi27LefnB8WNOXt31MEIO3xfJ4qk4D1zFakmUGn9NcfxOLUy0wktzd49_3I4yD1pks7eXlsr174HXckFeMlFFUOqNZHFY9UYwh4bsFHQXUpp9bJhkyS-Qv-w May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: e85ae209587a4c408afb7fd926922b39 May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:52:48.658286 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e85ae209587a4c408afb7fd926922b39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: e85ae209587a4c408afb7fd926922b39 May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-VolumesSnapshotsTestJSON-964541247-project May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: a06b95033ffd45cd9873c0b2618be401 May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-VolumesSnapshotsTestJSON-964541247-project May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Snapshot f442de9d-9b99-4cbb-b2a1-19513859aa54 could not be found. May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'f442de9d-9b99-4cbb-b2a1-19513859aa54'} May 25 16:52:48.660317 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot f442de9d-9b99-4cbb-b2a1-19513859aa54 could not be found. (HTTP 404) (Request-ID: req-fc23ed5d-4a6c-4311-82f9-2641b84c78e8) May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:48.661357 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:52:48.663078 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot f442de9d-9b99-4cbb-b2a1-19513859aa54 could not be found. May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show May 25 16:52:48.664511 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot f442de9d-9b99-4cbb-b2a1-19513859aa54 could not be found. May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot f442de9d-9b99-4cbb-b2a1-19513859aa54 could not be found. May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-b5825320-ebcd-4f2a-8b02-59528dbbb93e tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Returning 404 to user: Snapshot f442de9d-9b99-4cbb-b2a1-19513859aa54 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:48.666191 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-b5825320-ebcd-4f2a-8b02-59528dbbb93e tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54" status: 404 len: 111 microversion: 2.1 time: 0.033561 May 25 16:52:48.667332 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1328/2632] 10.222.0.45 () {58 vars in 1250 bytes} [Wed May 25 16:52:48 2022] GET /compute/v2.1/os-snapshots/f442de9d-9b99-4cbb-b2a1-19513859aa54 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:52:48.669985 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-fb8e34df-d391-466e-80be-903ed76feb28 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a3004f3a0>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:48.734269 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-fb8e34df-d391-466e-80be-903ed76feb28 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "DELETE /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588" status: 202 len: 0 microversion: 2.1 time: 0.068477 May 25 16:52:48.734864 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1305/2633] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:52:48 2022] DELETE /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 => generated 0 bytes in 70 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 25 16:52:48.743371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-78222cb0-766e-4af8-9e3d-8e3410a93dd5 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a30456c10>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:48.793480 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-78222cb0-766e-4af8-9e3d-8e3410a93dd5 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588" status: 200 len: 345 microversion: 2.1 time: 0.053921 May 25 16:52:48.794125 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1329/2634] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:48 2022] GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 => generated 345 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:52:49.807690 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f75c7037-73dc-4c7c-a9a3-3f1ac529e5df tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a236eb280>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:50.007802 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-f75c7037-73dc-4c7c-a9a3-3f1ac529e5df tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] HTTP exception thrown: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f75c7037-73dc-4c7c-a9a3-3f1ac529e5df tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Request method failure captured: May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 HTTP/1.1 May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijl7a60VPtxCXOQpP5Bw_iAG4nNiQPVedCqsBM0LZCksk2u4Kt1rS5juOi27LefnB8WNOXt31MEIO3xfJ4qk4D1zFakmUGn9NcfxOLUy0wktzd49_3I4yD1pks7eXlsr174HXckFeMlFFUOqNZHFY9UYwh4bsFHQXUpp9bJhkyS-Qv-w May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: e85ae209587a4c408afb7fd926922b39 May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:52:50.011197 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e85ae209587a4c408afb7fd926922b39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: e85ae209587a4c408afb7fd926922b39 May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesSnapshotsTestJSON-964541247-project May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: a06b95033ffd45cd9873c0b2618be401 May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesSnapshotsTestJSON-964541247-project May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'bfb8f712-4fb4-48dd-a53b-615bd89d4588'} May 25 16:52:50.014909 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi item = cinderclient( May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. (HTTP 404) (Request-ID: req-02fed918-a7a5-41bc-ad8d-f8bd14f80470) May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:50.016997 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi item = cinderclient( May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:52:50.019402 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 25 16:52:50.021851 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-f75c7037-73dc-4c7c-a9a3-3f1ac529e5df tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Returning 404 to user: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-f75c7037-73dc-4c7c-a9a3-3f1ac529e5df tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588" status: 404 len: 109 microversion: 2.1 time: 0.210765 May 25 16:52:50.025081 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1306/2635] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:49 2022] GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 => generated 109 bytes in 212 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:52:50.028198 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6f939f53-baca-40c0-b221-fb101d371372 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a23e44f70>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:50.057438 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-6f939f53-baca-40c0-b221-fb101d371372 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] HTTP exception thrown: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6f939f53-baca-40c0-b221-fb101d371372 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Request method failure captured: May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 HTTP/1.1 May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijl7a60VPtxCXOQpP5Bw_iAG4nNiQPVedCqsBM0LZCksk2u4Kt1rS5juOi27LefnB8WNOXt31MEIO3xfJ4qk4D1zFakmUGn9NcfxOLUy0wktzd49_3I4yD1pks7eXlsr174HXckFeMlFFUOqNZHFY9UYwh4bsFHQXUpp9bJhkyS-Qv-w May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: e85ae209587a4c408afb7fd926922b39 May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:52:50.059412 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e85ae209587a4c408afb7fd926922b39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: e85ae209587a4c408afb7fd926922b39 May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-VolumesSnapshotsTestJSON-964541247-project May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: a06b95033ffd45cd9873c0b2618be401 May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-VolumesSnapshotsTestJSON-964541247-project May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': 'bfb8f712-4fb4-48dd-a53b-615bd89d4588'} May 25 16:52:50.061259 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. (HTTP 404) (Request-ID: req-8f6aef5a-f83a-4d53-a85d-d4cfd39c930b) May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:50.062235 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 25 16:52:50.063607 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 25 16:52:50.064907 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-6f939f53-baca-40c0-b221-fb101d371372 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Returning 404 to user: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-6f939f53-baca-40c0-b221-fb101d371372 tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "DELETE /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588" status: 404 len: 109 microversion: 2.1 time: 0.036736 May 25 16:52:50.066556 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1330/2636] 10.222.0.45 () {58 vars in 1247 bytes} [Wed May 25 16:52:50 2022] DELETE /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:52:50.074297 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4cde8488-b766-46c0-88f8-992e8df12aad tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Calling method '.version_select at 0x7f4a236eb160>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:52:50.102114 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-4cde8488-b766-46c0-88f8-992e8df12aad tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] HTTP exception thrown: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4cde8488-b766-46c0-88f8-992e8df12aad tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Request method failure captured: May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 HTTP/1.1 May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijl7a60VPtxCXOQpP5Bw_iAG4nNiQPVedCqsBM0LZCksk2u4Kt1rS5juOi27LefnB8WNOXt31MEIO3xfJ4qk4D1zFakmUGn9NcfxOLUy0wktzd49_3I4yD1pks7eXlsr174HXckFeMlFFUOqNZHFY9UYwh4bsFHQXUpp9bJhkyS-Qv-w May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: e85ae209587a4c408afb7fd926922b39 May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:52:50.104127 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_e85ae209587a4c408afb7fd926922b39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/e85ae209587a4c408afb7fd926922b39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: e85ae209587a4c408afb7fd926922b39 May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-964541247 May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesSnapshotsTestJSON-964541247-project May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: a06b95033ffd45cd9873c0b2618be401 May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesSnapshotsTestJSON-964541247-project May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': 'bfb8f712-4fb4-48dd-a53b-615bd89d4588'} May 25 16:52:50.105768 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi item = cinderclient( May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. (HTTP 404) (Request-ID: req-147cb667-384d-4670-a02b-4f1a505ef68c) May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:50.106804 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi item = cinderclient( May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 25 16:52:50.108457 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 25 16:52:50.109969 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4cde8488-b766-46c0-88f8-992e8df12aad tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] Returning 404 to user: Volume bfb8f712-4fb4-48dd-a53b-615bd89d4588 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4cde8488-b766-46c0-88f8-992e8df12aad tempest-VolumesSnapshotsTestJSON-964541247 tempest-VolumesSnapshotsTestJSON-964541247-project] 10.222.0.45 "GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588" status: 404 len: 109 microversion: 2.1 time: 0.035525 May 25 16:52:50.111543 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1307/2637] 10.222.0.45 () {58 vars in 1244 bytes} [Wed May 25 16:52:50 2022] GET /compute/v2.1/os-volumes/bfb8f712-4fb4-48dd-a53b-615bd89d4588 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:54:18.841458 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:54:18.842337 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:54:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:54:19.005042 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1219784686", "imageRef": "60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a", "flavorRef": "452"}} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:54:19.057316 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.network.neutron [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] validate_networks() for None {{(pid=117795) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 25 16:54:19.333662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Flavor limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 25 16:54:19.334232 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Image limits 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 25 16:54:19.334921 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Flavor pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 25 16:54:19.335430 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Image pref 0:0:0 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 25 16:54:19.335841 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.virt.hardware [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117795) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 25 16:54:19.344905 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:19.345435 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:19.346110 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:19.346567 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:19.364139 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Getting quotas for project f4ec46474b514eab977d587d3e592353. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 25 16:54:19.368401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.quota [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Getting quotas for user a2c55c9acb624f168a4090e6ed1e8abd and project f4ec46474b514eab977d587d3e592353. Resources: {'ram', 'cores', 'instances'} {{(pid=117795) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 25 16:54:19.379813 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Going to run 1 instances... {{(pid=117795) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 25 16:54:19.382214 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='60ec5353-0cac-4bcd-8fe8-ca0d2703bd1a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117795) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 25 16:54:19.435713 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-45da4180-c8ed-4588-9196-8604fd3efdff tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.436036 May 25 16:54:19.436408 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1331/2638] 10.222.0.45 () {60 vars in 1146 bytes} [Wed May 25 16:54:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 437 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 25 16:54:19.450848 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-54a6f4b8-b833-49d8-9c91-0b4ff8f3b0d9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:19.451619 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-54a6f4b8-b833-49d8-9c91-0b4ff8f3b0d9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:19.511807 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-54a6f4b8-b833-49d8-9c91-0b4ff8f3b0d9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:19.520465 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-54a6f4b8-b833-49d8-9c91-0b4ff8f3b0d9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1215 microversion: 2.1 time: 0.077779 May 25 16:54:19.521034 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1308/2639] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:19 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1215 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:20.539205 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-774ed04f-c85f-42d7-88f9-fb2347da3d91 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:20.540054 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-774ed04f-c85f-42d7-88f9-fb2347da3d91 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:20.546674 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-774ed04f-c85f-42d7-88f9-fb2347da3d91 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:20.547305 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-774ed04f-c85f-42d7-88f9-fb2347da3d91 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:20.548066 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-774ed04f-c85f-42d7-88f9-fb2347da3d91 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:20.548651 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-774ed04f-c85f-42d7-88f9-fb2347da3d91 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:20.656265 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-774ed04f-c85f-42d7-88f9-fb2347da3d91 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:20.666732 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-774ed04f-c85f-42d7-88f9-fb2347da3d91 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1285 microversion: 2.1 time: 0.132697 May 25 16:54:20.667445 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1332/2640] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:20 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1285 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:21.685846 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0f472539-afa5-4556-b433-7fc07806c6f6 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:21.686360 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0f472539-afa5-4556-b433-7fc07806c6f6 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:21.692898 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0f472539-afa5-4556-b433-7fc07806c6f6 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:21.693341 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0f472539-afa5-4556-b433-7fc07806c6f6 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:21.694088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0f472539-afa5-4556-b433-7fc07806c6f6 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:21.694513 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0f472539-afa5-4556-b433-7fc07806c6f6 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:21.900062 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0f472539-afa5-4556-b433-7fc07806c6f6 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:21.907982 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0f472539-afa5-4556-b433-7fc07806c6f6 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1315 microversion: 2.1 time: 0.226078 May 25 16:54:21.908557 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1309/2641] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:21 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1315 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:22.420603 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-e79de381-a5a5-4c47-9849-031cd43c8150 req-5fab7816-121b-4040-986e-4602afb57e9e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "402e0205-0bff-4b9e-bc84-16c1356fad58", "tag": "8acd1631-7b07-4562-9b7c-e359d26574bc"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:54:22.445659 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-e79de381-a5a5-4c47-9849-031cd43c8150 req-5fab7816-121b-4040-986e-4602afb57e9e service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:22.446252 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-e79de381-a5a5-4c47-9849-031cd43c8150 req-5fab7816-121b-4040-986e-4602afb57e9e service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:22.458953 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-e79de381-a5a5-4c47-9849-031cd43c8150 req-5fab7816-121b-4040-986e-4602afb57e9e service nova] Creating event network-changed:8acd1631-7b07-4562-9b7c-e359d26574bc for instance 402e0205-0bff-4b9e-bc84-16c1356fad58 on hv-cinder-84327 May 25 16:54:22.465401 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-e79de381-a5a5-4c47-9849-031cd43c8150 req-5fab7816-121b-4040-986e-4602afb57e9e service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052441 May 25 16:54:22.466230 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1333/2642] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:54:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:54:22.930074 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-af34c2a7-1800-4ff7-b94c-befad8520107 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:22.931080 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-af34c2a7-1800-4ff7-b94c-befad8520107 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:22.942108 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-af34c2a7-1800-4ff7-b94c-befad8520107 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:22.942861 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-af34c2a7-1800-4ff7-b94c-befad8520107 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:22.944231 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-af34c2a7-1800-4ff7-b94c-befad8520107 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:22.944911 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-af34c2a7-1800-4ff7-b94c-befad8520107 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:23.101945 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-af34c2a7-1800-4ff7-b94c-befad8520107 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:23.109770 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-af34c2a7-1800-4ff7-b94c-befad8520107 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1315 microversion: 2.1 time: 0.186529 May 25 16:54:23.110342 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1310/2643] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:22 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1315 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:24.127020 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-875fe0cf-c5ea-406a-9ead-9048669bc0a0 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:24.127755 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-875fe0cf-c5ea-406a-9ead-9048669bc0a0 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:24.135832 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-875fe0cf-c5ea-406a-9ead-9048669bc0a0 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:24.136388 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-875fe0cf-c5ea-406a-9ead-9048669bc0a0 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:24.137218 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-875fe0cf-c5ea-406a-9ead-9048669bc0a0 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:24.137758 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-875fe0cf-c5ea-406a-9ead-9048669bc0a0 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:24.275761 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-875fe0cf-c5ea-406a-9ead-9048669bc0a0 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:24.283416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-875fe0cf-c5ea-406a-9ead-9048669bc0a0 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1315 microversion: 2.1 time: 0.161003 May 25 16:54:24.283949 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1334/2644] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:24 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:25.303486 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-dce20f52-f36a-4c16-b0cd-0432682434e9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:25.304188 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-dce20f52-f36a-4c16-b0cd-0432682434e9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:25.312319 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dce20f52-f36a-4c16-b0cd-0432682434e9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:25.312772 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dce20f52-f36a-4c16-b0cd-0432682434e9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:25.313516 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dce20f52-f36a-4c16-b0cd-0432682434e9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:25.314003 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-dce20f52-f36a-4c16-b0cd-0432682434e9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:25.451478 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-dce20f52-f36a-4c16-b0cd-0432682434e9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:25.459294 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-dce20f52-f36a-4c16-b0cd-0432682434e9 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1315 microversion: 2.1 time: 0.162031 May 25 16:54:25.459896 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1311/2645] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:25 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1315 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:26.480971 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ee49fae1-c5ee-4d2d-8058-1aa4b80baf16 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:26.481629 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ee49fae1-c5ee-4d2d-8058-1aa4b80baf16 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:26.488329 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee49fae1-c5ee-4d2d-8058-1aa4b80baf16 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:26.488782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee49fae1-c5ee-4d2d-8058-1aa4b80baf16 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:26.489545 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee49fae1-c5ee-4d2d-8058-1aa4b80baf16 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:26.489984 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ee49fae1-c5ee-4d2d-8058-1aa4b80baf16 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:26.626653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ee49fae1-c5ee-4d2d-8058-1aa4b80baf16 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:26.635441 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ee49fae1-c5ee-4d2d-8058-1aa4b80baf16 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1315 microversion: 2.1 time: 0.160742 May 25 16:54:26.636400 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1335/2646] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:26 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1315 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:27.653631 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-b4e00de1-b8bd-4bcd-95dd-77a5cc275fcb tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:27.654270 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-b4e00de1-b8bd-4bcd-95dd-77a5cc275fcb tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:27.661302 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4e00de1-b8bd-4bcd-95dd-77a5cc275fcb tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:27.661753 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4e00de1-b8bd-4bcd-95dd-77a5cc275fcb tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:27.662820 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4e00de1-b8bd-4bcd-95dd-77a5cc275fcb tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:27.663277 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-b4e00de1-b8bd-4bcd-95dd-77a5cc275fcb tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:27.810729 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-b4e00de1-b8bd-4bcd-95dd-77a5cc275fcb tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:27.818163 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-b4e00de1-b8bd-4bcd-95dd-77a5cc275fcb tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1315 microversion: 2.1 time: 0.168548 May 25 16:54:27.819066 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1312/2647] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:27 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1315 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:28.464056 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [req-a4f1c30a-a5ac-445f-b22f-13ce0338f439 req-544ad39c-4674-481e-a884-3680cefc0806 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "402e0205-0bff-4b9e-bc84-16c1356fad58", "name": "network-vif-plugged", "status": "completed", "tag": "8acd1631-7b07-4562-9b7c-e359d26574bc"}]} {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:54:28.474446 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-a4f1c30a-a5ac-445f-b22f-13ce0338f439 req-544ad39c-4674-481e-a884-3680cefc0806 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:28.474963 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [req-a4f1c30a-a5ac-445f-b22f-13ce0338f439 req-544ad39c-4674-481e-a884-3680cefc0806 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:28.487082 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.compute.server_external_events [req-a4f1c30a-a5ac-445f-b22f-13ce0338f439 req-544ad39c-4674-481e-a884-3680cefc0806 service nova] Creating event network-vif-plugged:8acd1631-7b07-4562-9b7c-e359d26574bc for instance 402e0205-0bff-4b9e-bc84-16c1356fad58 on hv-cinder-84327 May 25 16:54:28.492324 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [req-a4f1c30a-a5ac-445f-b22f-13ce0338f439 req-544ad39c-4674-481e-a884-3680cefc0806 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.155407 May 25 16:54:28.492956 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1336/2648] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:54:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:54:28.836661 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-324ad082-da04-48f7-a070-774820358023 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:28.837276 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-324ad082-da04-48f7-a070-774820358023 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:28.845875 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-324ad082-da04-48f7-a070-774820358023 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:28.847088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-324ad082-da04-48f7-a070-774820358023 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:28.848157 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-324ad082-da04-48f7-a070-774820358023 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:28.848602 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-324ad082-da04-48f7-a070-774820358023 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:28.988128 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-324ad082-da04-48f7-a070-774820358023 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:28.997984 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-324ad082-da04-48f7-a070-774820358023 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1315 microversion: 2.1 time: 0.166082 May 25 16:54:28.998600 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1313/2649] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:28 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1315 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:30.013657 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=2 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:54:30.014403 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG dbcounter [-] [117795] Writing DB stats nova_cell0:SELECT=6 {{(pid=117795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:54:30.018826 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-07cbfa55-9683-4fc6-ad21-d4eea83f0574 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:30.019581 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-07cbfa55-9683-4fc6-ad21-d4eea83f0574 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:30.026982 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07cbfa55-9683-4fc6-ad21-d4eea83f0574 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:30.027443 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07cbfa55-9683-4fc6-ad21-d4eea83f0574 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:30.028203 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07cbfa55-9683-4fc6-ad21-d4eea83f0574 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:30.028628 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-07cbfa55-9683-4fc6-ad21-d4eea83f0574 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:30.169386 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-07cbfa55-9683-4fc6-ad21-d4eea83f0574 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:30.177411 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-07cbfa55-9683-4fc6-ad21-d4eea83f0574 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1315 microversion: 2.1 time: 0.165943 May 25 16:54:30.177977 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1337/2650] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:30 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1315 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:31.195261 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-253aa687-5863-4d31-a9e3-44be9b8d22d8 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:31.195839 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-253aa687-5863-4d31-a9e3-44be9b8d22d8 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:31.198132 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG dbcounter [-] [117794] Writing DB stats nova_cell0:SELECT=2 {{(pid=117794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:54:31.204088 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-253aa687-5863-4d31-a9e3-44be9b8d22d8 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:31.204540 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-253aa687-5863-4d31-a9e3-44be9b8d22d8 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:31.205293 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-253aa687-5863-4d31-a9e3-44be9b8d22d8 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:31.205710 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-253aa687-5863-4d31-a9e3-44be9b8d22d8 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:31.341605 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-253aa687-5863-4d31-a9e3-44be9b8d22d8 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:31.349528 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-253aa687-5863-4d31-a9e3-44be9b8d22d8 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1465 microversion: 2.1 time: 0.159047 May 25 16:54:31.350118 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1314/2651] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:31 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1465 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:31.392215 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-1640d56e-daf5-4b64-aacf-cdceb96c6fe3 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:31.392782 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1640d56e-daf5-4b64-aacf-cdceb96c6fe3 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:31.399618 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1640d56e-daf5-4b64-aacf-cdceb96c6fe3 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:31.400064 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1640d56e-daf5-4b64-aacf-cdceb96c6fe3 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:31.400821 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1640d56e-daf5-4b64-aacf-cdceb96c6fe3 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:31.401247 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-1640d56e-daf5-4b64-aacf-cdceb96c6fe3 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:31.426594 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-1640d56e-daf5-4b64-aacf-cdceb96c6fe3 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Going to try to terminate instance {{(pid=117795) delete /opt/stack/nova/nova/compute/api.py:2664}} May 25 16:54:31.507181 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-1640d56e-daf5-4b64-aacf-cdceb96c6fe3 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "DELETE /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 204 len: 0 microversion: 2.1 time: 0.118907 May 25 16:54:31.507847 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1338/2652] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:54:31 2022] DELETE /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 25 16:54:31.516490 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-0cfe1d92-92b9-4ba8-90d9-de19dc5f180e tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:31.517096 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-0cfe1d92-92b9-4ba8-90d9-de19dc5f180e tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:31.523312 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cfe1d92-92b9-4ba8-90d9-de19dc5f180e tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:31.523752 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cfe1d92-92b9-4ba8-90d9-de19dc5f180e tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:31.524500 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cfe1d92-92b9-4ba8-90d9-de19dc5f180e tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:31.524928 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-0cfe1d92-92b9-4ba8-90d9-de19dc5f180e tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:31.656613 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.policy [None req-0cfe1d92-92b9-4ba8-90d9-de19dc5f180e tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117794) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:31.666008 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-0cfe1d92-92b9-4ba8-90d9-de19dc5f180e tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1471 microversion: 2.1 time: 0.153260 May 25 16:54:31.666785 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1315/2653] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:31 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1471 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 25 16:54:32.689071 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-ea461994-4c4b-4e0b-b58e-25a86f409c22 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:32.689973 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-ea461994-4c4b-4e0b-b58e-25a86f409c22 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:32.699662 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea461994-4c4b-4e0b-b58e-25a86f409c22 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:32.700159 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea461994-4c4b-4e0b-b58e-25a86f409c22 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:32.700911 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea461994-4c4b-4e0b-b58e-25a86f409c22 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:32.701351 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-ea461994-4c4b-4e0b-b58e-25a86f409c22 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:32.842718 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-ea461994-4c4b-4e0b-b58e-25a86f409c22 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:32.851120 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-ea461994-4c4b-4e0b-b58e-25a86f409c22 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1471 microversion: 2.1 time: 0.168238 May 25 16:54:32.851735 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1339/2654] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:32 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1471 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:33.429275 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [req-7abbfb46-23b8-4f89-a550-2358962214ff req-697513db-069f-47c3-b406-b18346ae2740 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "402e0205-0bff-4b9e-bc84-16c1356fad58", "name": "network-vif-deleted", "tag": "8acd1631-7b07-4562-9b7c-e359d26574bc"}]} {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 25 16:54:33.438542 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-7abbfb46-23b8-4f89-a550-2358962214ff req-697513db-069f-47c3-b406-b18346ae2740 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:33.438774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [req-7abbfb46-23b8-4f89-a550-2358962214ff req-697513db-069f-47c3-b406-b18346ae2740 service nova] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:33.451526 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.compute.server_external_events [req-7abbfb46-23b8-4f89-a550-2358962214ff req-697513db-069f-47c3-b406-b18346ae2740 service nova] Creating event network-vif-deleted:8acd1631-7b07-4562-9b7c-e359d26574bc for instance 402e0205-0bff-4b9e-bc84-16c1356fad58 on hv-cinder-84327 May 25 16:54:33.457011 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [req-7abbfb46-23b8-4f89-a550-2358962214ff req-697513db-069f-47c3-b406-b18346ae2740 service nova] 10.222.0.45 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031748 May 25 16:54:33.457608 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1316/2655] 10.222.0.45 () {64 vars in 1317 bytes} [Wed May 25 16:54:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 25 16:54:33.871000 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:33.872342 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:33.879031 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:33.879520 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:33.880281 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:33.880729 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:33.915795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:33.915795 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:33.916179 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.objects.instance [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lazy-loading 'fault' on Instance uuid 402e0205-0bff-4b9e-bc84-16c1356fad58 {{(pid=117795) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 25 16:54:33.965879 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.policy [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2c55c9acb624f168a4090e6ed1e8abd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ec46474b514eab977d587d3e592353', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117795) authorize /opt/stack/nova/nova/policy.py:199}} May 25 16:54:33.972770 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-59d83945-1562-4494-88cb-d0d9a06d58a1 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 200 len: 1305 microversion: 2.1 time: 0.105677 May 25 16:54:33.973333 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1340/2656] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:33 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 1305 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 25 16:54:34.995649 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:34.996354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] [instance: 402e0205-0bff-4b9e-bc84-16c1356fad58] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:35.002774 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:35.003241 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:35.004025 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:35.004437 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:35.018042 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] HTTP exception thrown: Instance 402e0205-0bff-4b9e-bc84-16c1356fad58 could not be found. May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Request method failure captured: May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 HTTP/1.1 May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijl85niiMNnqiXM6o6GTTJjINJwr937rqqAksg2APXtfbYJaYrKxg1mOciii4EM2keqietc5lylg_vP9yfq5Q-YCYjWqXCXNHWjQVTdBgcyFaU48V74yOmi-EHONuvUdZuOTERG-sBd0SPL2OX51BAVigYWX5-qaiD3XiyTojjE1PZqs May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: f4ec46474b514eab977d587d3e592353 May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-VolumesNegativeTest-1393246282 May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:54:35.019722 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_f4ec46474b514eab977d587d3e592353"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/f4ec46474b514eab977d587d3e592353"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/f4ec46474b514eab977d587d3e592353"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/f4ec46474b514eab977d587d3e592353"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-VolumesNegativeTest-1393246282 May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: f4ec46474b514eab977d587d3e592353 May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-VolumesNegativeTest-1393246282 May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-VolumesNegativeTest-1393246282-project May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: a2c55c9acb624f168a4090e6ed1e8abd May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-VolumesNegativeTest-1393246282-project May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 402e0205-0bff-4b9e-bc84-16c1356fad58 could not be found. May 25 16:54:35.021095 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '402e0205-0bff-4b9e-bc84-16c1356fad58'} May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 402e0205-0bff-4b9e-bc84-16c1356fad58 could not be found. May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:54:35.021808 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 402e0205-0bff-4b9e-bc84-16c1356fad58 could not be found. May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 402e0205-0bff-4b9e-bc84-16c1356fad58 could not be found. May 25 16:54:35.023112 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:35.024242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] Returning 404 to user: Instance 402e0205-0bff-4b9e-bc84-16c1356fad58 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:54:35.024242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-4aedfdc6-2a6d-451d-9195-de0334b96870 tempest-VolumesNegativeTest-1393246282 tempest-VolumesNegativeTest-1393246282-project] 10.222.0.45 "GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58" status: 404 len: 111 microversion: 2.1 time: 0.033350 May 25 16:54:35.024242 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1317/2657] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:34 2022] GET /compute/v2.1/servers/402e0205-0bff-4b9e-bc84-16c1356fad58 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:54:37.336200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:37.336812 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.compute.api [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117795) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:37.344483 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:37.344988 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:37.345764 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:37.346200 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG oslo_concurrency.lockutils [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117795) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:37.360566 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.wsgi [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: request: DELETE /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 HTTP/1.1 May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept: application/json May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Accept-Encoding: identity May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Connection: close May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Length: 0 May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Content-Type: application/json May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Host: 10.222.0.45 May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Openstack-System-Scope: None May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: User-Agent: python-urllib3/1.26.9 May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Id: None May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Domain-Name: None May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Identity-Status: Confirmed May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Is-Admin-Project: True May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Id: default May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Domain-Name: Default May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Role: reader,member May 25 16:54:37.362241 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Roles: reader,member May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Id: default May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Domain-Name: Default May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: method: > May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: exception: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.363653 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: action_args: {'id': '48341208-161c-454c-b88e-56a30f695cb7'} May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: {{(pid=117795) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:54:37.364349 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.365416 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: ERROR nova.api.openstack.wsgi May 25 16:54:37.366666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: DEBUG nova.api.openstack.wsgi [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. {{(pid=117795) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:54:37.366666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO nova.api.openstack.requestlog [None req-d2156aa9-d63c-4f20-9e21-0271d809c683 tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "DELETE /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 404 len: 111 microversion: 2.1 time: 0.031399 May 25 16:54:37.366666 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: [pid: 117795|app: 0|req: 1341/2658] 10.222.0.45 () {58 vars in 1238 bytes} [Wed May 25 16:54:37 2022] DELETE /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:54:37.372223 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Calling method '>' {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 25 16:54:37.372845 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.compute.api [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] [instance: 48341208-161c-454c-b88e-56a30f695cb7] Fetching instance by UUID {{(pid=117794) get /opt/stack/nova/nova/compute/api.py:2920}} May 25 16:54:37.379354 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:37.379799 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:37.380554 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 25 16:54:37.380958 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG oslo_concurrency.lockutils [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Lock "d6825671-c3c0-4b92-ba50-876e3f2196e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117794) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 25 16:54:37.394477 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.wsgi [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] HTTP exception thrown: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Request method failure captured: May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: request: GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 HTTP/1.1 May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept: application/json May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Accept-Encoding: identity May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Connection: close May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Length: 0 May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Content-Type: application/json May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Host: 10.222.0.45 May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Openstack-System-Scope: None May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: User-Agent: python-urllib3/1.26.9 May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Auth-Token: gAAAAABijlaZ2rhNslI7qI5ROGWONTQCc-V2IAJ_bdVhpLePiaBsveeSlb13yJlia3XdyrCayfLOU9B5obGRpgDQQ4t3wxGbPzUt_QGysSXPq_NeuCi0GY7kwYGTf3wR8xPcJMFKn1VewjDgJxtGl1LnrwPTJg1X1aPPia-Rhdh11XjHKd05fHA May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Id: None May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Domain-Name: None May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Identity-Status: Confirmed May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Is-Admin-Project: True May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Id: default May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Domain-Name: Default May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Project-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Role: reader,member May 25 16:54:37.396036 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Roles: reader,member May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.45:8080", "publicURL": "http://10.222.0.45:8080/v1/AUTH_ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/volume/v3/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2/ec1d676becac4112a7c0c319e54b2cd2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.45/compute/v2.1"}]}] May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant: tempest-AttachVolumeNegativeTest-568938212 May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Id: ec1d676becac4112a7c0c319e54b2cd2 May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-568938212 May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Id: default May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Domain-Name: Default May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Id: a0c9f3f10fd142439521e4b36cf33157 May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: X-User-Name: tempest-AttachVolumeNegativeTest-568938212-project May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: method: > May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: exception: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.397626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: action_args: {'id': '48341208-161c-454c-b88e-56a30f695cb7'} May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: {{(pid=117794) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 25 16:54:37.398372 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. May 25 16:54:37.399638 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: ERROR nova.api.openstack.wsgi May 25 16:54:37.400795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: DEBUG nova.api.openstack.wsgi [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] Returning 404 to user: Instance 48341208-161c-454c-b88e-56a30f695cb7 could not be found. {{(pid=117794) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 25 16:54:37.400795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO nova.api.openstack.requestlog [None req-26a9cc86-517b-4602-ab56-d78d6f997a7b tempest-AttachVolumeNegativeTest-568938212 tempest-AttachVolumeNegativeTest-568938212-project] 10.222.0.45 "GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7" status: 404 len: 111 microversion: 2.1 time: 0.029078 May 25 16:54:37.400795 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: [pid: 117794|app: 0|req: 1318/2659] 10.222.0.45 () {58 vars in 1235 bytes} [Wed May 25 16:54:37 2022] GET /compute/v2.1/servers/48341208-161c-454c-b88e-56a30f695cb7 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 25 16:55:43.392712 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:55:43.393312 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:55:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:55:44.107396 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:55:44.107878 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:55:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:57:13.678099 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:57:13.679041 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:57:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:57:31.101178 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:57:31.101626 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:57:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:57:38.855353 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:57:38.856445 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:57:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:57:43.489601 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:57:43.490481 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:57:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:59:03.402690 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:59:03.403843 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 16:59:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 16:59:04.118659 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 16:59:04.119195 dv-cinder-843279-2-iscsi devstack@n-api.service[117794]: Wed May 25 16:59:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 25 17:00:33.691768 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 25 17:00:33.692371 dv-cinder-843279-2-iscsi devstack@n-api.service[117795]: Wed May 25 17:00:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!